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EXECUTIVE  SUMMARY 


1.0  Introduction 

This  summary  serves  as  a  stand-alone  document,  as  well  as  part  of  this  report.  For  that  reason, 
the  reader  wih  find  some  duplication  of  verbiage  and  figures  between  the  summary  and  the  full 
report. 

2.0  JADS  Overview 

The  Joint  Advanced  Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E)  was 
chartered  by  the  deputy  director.  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation),  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  in  October 
1994  to  investigate  the  utility  of  advanced  distributed  simulation  (ADS)  technologies  for  support 
of  developmental  test  and  evaluation  (DT&E)  and  operational  test  and  evaluation  (OT&E).  The 
program  is  Air  Force  led  with  Army  and  Navy  participation.  JADS  Joint  Test  Force  (JTF) 
manning  currently  includes  18  Air  Force,  4  Air  Force  civilians,  12  Army,  and  1  Navy  civilian. 
Science  Applications  International  Corporation  and  the  Georgia  Tech  Research  Institute  provide 
contracted  technical  support.  The  program  is  currently  scheduled  to  end  in  March  2000. 

The  JADS  JTF  is  directly  investigating  ADS  applications  in  three  slices  of  the  test  and  evaluation 
(T&E)  spectrum:  the  System  Integration  Test  (SIT)  which  explored  ADS  support  of  air-to-air 
missile  testing;  the  End-to-End  (ETE)  Test  which  is  investigating  ADS  support  for  command, 
control,  communications,  computers,  intelligence,  surveillance,  and  reconnaissance  (C4ISR) 
testing;  and  the  Electronic  Warfare  (EW)  Test  which  is  exploring  ADS  support  for  EW  testing. 
The  JTF  is  also  chartered  to  observe  or  participate  at  a  modest  level  in  ADS  activities  sponsored 
and  conducted  by  other  agencies  in  an  effort  to  broaden  conclusions  developed  in  the  three 
dedicated  test  areas. 

Phase  2,  the  laboratory  test,  of  the  ETE  Test  is  the  subject  of  this  summary  report. 

3.0  ETE  Test  Overview 

The  ETE  Test  is  designed  to  evaluate  the  utility  of  ADS  to  support  testing  of  C4ISR  systems. 
The  test  uses  the  Joint  Surveillance  Target  Attack  Radar  System  (Joint  STARS)  as  one 
component  of  a  representative  C4ISR  system.  The  ETE  Test  also  evaluates  the  capability  of  the 
JADS  Test  Control  and  Analysis  Center  (TCAC)  to  control  a  distributed  test  of  this  type  and 
remotely  monitor  and  analyze  test  results. 

The  ETE  Test  consists  of  four  phases.  Phase  1  developed  or  modified  the  components  needed  to 
develop  the  ADS  test  environment.  Phase  2  used  the  ADS  test  environment  to  evaluate  the  utility 
of  ADS  to  support  DT&E  and  early  OT&E  of  a  C4ISR  system  in  a  laboratory  environment. 
Phase  3  transitions  portions  of  the  architecture  to  the  E-8C  aircraft,  ensures  that  the  components 
function  properly,  and  checks  that  the  synthetic  environment  interacts  properly  with  the  aircraft 
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and  actual  light  ground  station  module  (LGSM).  Phase  4  wiU  evaluate  the  ability  to  perform  test 
and  evaluation  of  the  E-8C  and  LGSM  in  a  synthetically  enhanced  live  test  environment. 

4.0  Overview  of  ETE  Test  Phase  2 
4.1  Purpose 

Phase  2  determined  the  utility  of  ADS  to  support  DT&E  and  early  OT&E  of  a  C4ISR  system  in  a 
laboratory  environment.  Using  the  components  developed  in  Phase  1,  a  distributed  interactive 
simulation  (DIS)  network  with  appropriate  C4ISR  and  weapon  system  nodes  was  developed  to 
evaluate  a  representative  sensor-to-shooter  process.  A  virtual  Army  Tactical  Missile  System 
(ATACMS)  battalion  (Bn)  was  used  as  the  engagement  system.  The  test  objectives  were 

JADS  Issue  1.  What  is  the  present  utility  of  ADS,  including  DIS,  for  T&E? 

JADS  Objective  1-1.  Assess  the  validity  of  data  from  tests  utilizing  ADS,  including  DIS, 
during  test  execution. 

JADS  Objective  1-2.  Assess  the  benefits  of  using  ADS,  including  DIS,  in  T&E.  This  test 
objective  was  broken  down  into  subobjectives.  (Subobjective  1-2-1  is  not  applicable  to  the 
ETE  Test  Phase  2.) 

JADS  Subobjective  1-2-2.  Assess  ADS  capability  to  support  T&E  planning  and  test 
rehearsal. 

JADS  Subobjective  1-2-3.  Assess  ADS  capability  to  support  T&E  shortfalls. 

JADS  Issue  2.  What  are  the  critical  constraints,  concerns,  and  methodologies  when  using  ADS 
for  T&E? 

JADS  Objective  2-1.  Assess  the  critical  constraints  and  concerns  in  ADS  performance  for 
T&E.  This  objective  was  broken  down  into  subobjectives.  (Subobjective  2-1-1  is  not 
applicable  to  the  ETE  Test  Phase  2.) 

JADS  Subobjective  2-1-2.  Assess  network  and  communications  performance  constraints 
and  limitations. 

JADS  Subobjective  2-1-3.  Assess  the  impact  of  ADS  reliability,  availability,  and 
maintainability  on  T&E. 

JADS  Objective  2-2.  Assess  the  critical  constraints  and  concerns  in  ADS  support  systems 
for  T&E.  This  objective  was  broken  down  into  subobjectives. 
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JADS  Subobjective  2-2-1.  Assess  the  critical  constraints  and  concerns  regarding  ADS 
data  management  and  analysis  systems. 

JADS  Subobjective  2-2-2.  Assess  the  critical  constraints  and  concerns  regarding 
configuration  management  of  ADS  test  assets. 

JADS  Objective  2-3.  Develop  and  assess  methodologies  associated  with  ADS  for  T&E. 
(Subobjective  2-3-1  is  not  applicable  to  the  ETE  Test  Phase  2.) 

JADS  Subobjective  2-3-2.  Develop  and  assess  methodologies  associated  with  test 
execution  and  control  for  tests  using  ADS. 

4.2  Approach 

Figure  ES-1  provides  an  overview  of  the  ETE  Test  synthetic  environment. 


TCAC 

Albuquerque,  New  Mexico 


SCDL 


Grumman  Aerospace  Labs 
E-8C  Simulation 

Melbourne,  Florida 


AFATDS  =  Advanced  Field  Artillery  Tactical  Data  System  Janus  =  interactive,  computer-based  simulation  of  combat 

operationsSCDL  =  surveillance  control  data  link 

T-1  =  digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1 .544  megabits  per  second 

TAC  =  target  analysis  cell  TRAC  -  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center 

WSMR  =  White  Sands  Missile  Range,  New  Mexico 

Figure  ES-1.  ETE  Test  Phase  2  Synthetic  Environment 


The  ETE  Test  used  the  Janus  6.88D  simulation  to  generate  the  entities  representing  the  elements 
in  the  rear  of  a  threat  force.  The  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center 
(TRAC)  at  White  Sands  Missile  Range  (WSMR),  New  Mexico,  provided  the  Janus  scenario  feed. 

The  Test  Control  and  Analysis  Center,  in  Albuquerque,  New  Mexico,  provided  test  control. 

The  Joint  STARS  E-8C  simulation,  called  the  Virtual  Surveillance  Target  Attack  Radar  System 
(VSTARS),  represented  the  radar  subsystem  of  the  Joint  STARS  E-8C  in  a  laboratory 
environment.  It  was  composed  of  a  distributed  interactive  simulation  network  interface  unit 
(NIU),  a  radar  processor  simulator  and  integrator  (RPSI)  that  contained  the  two  real-time  radar 
simulations  with  necessary  databases,  and  various  simulations  of  E-8C  processes.  Figure  ES-2 
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provides  more  information  on  the  VSTARS  architecture.  VSTARS  was  operated  at  the  Northrop 
Grumman  Surveillance  and  Battle  Management  Systems  facility  in  Melbourne,  Florida. 
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CDP  -  central  data  processor  M&IS  -  management  and  integration  software 

SM&C  -  system  management  and  control  SMO  -  system  management  officer 

T-1  =  digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1 .544  megabits  per  second 


Figure  ES-2.  VSTARS  Architecture 

The  LGSM  and  target  analysis  cell  (TAG)  were  represented  by  Bravo  Company,  303d  Military 
Intelligence  Battalion.  Fire  support,  in  the  form  of  the  Advanced  Field  Artillery  Tactical  Data 
System  (AFATDS),  was  represented  by  soldiers  from  the  4th  Infantry  Division  (Mechanized). 

Communications  among  these  command,  control,  communications,  computers  and  intelligence 
(C4I)  systems  employed  such  doctrinaUy  correct  means  as  the  CGS-100,  a  subsystem  of  the 
Compartmented  All  Source  Analysis  System  (ASAS)  Message  Processing  System  (CAMPS), 
remote  workstations  (RWSs),  and  AFATDS  message  traffic. 

The  Tactical  Army  Fire  Support  Model  (TAFSM)  simulation  modeled  the  Army  Tactical  Missile 
System  (ATACMS)  battahon  and  sent  the  fire  and  detonate  protocol  data  units  (PDUs)  to  the 
Janus  6.88D  simulation.  Janus  then  modeled  the  engagement  results  and  reflected  them  in  the 
synthetic  environment. 
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5.0  ETE  Test  Phase  2  Results 


5.1  Schedule 

The  overall  ETE  Test  schedule  is  presented  in  Figure  ES-3.  Phase  2  testing  proceeded  as 
scheduled. 
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Figure  ES-3.  ETE  Test  Schedule 


5.2  Fulfillment  of  Test  Objectives 

All  ETE  Test  Phase  2  objectives  were  met.  The  ETE  Test  team  determined  that  ADS  testing  can 
be  beneficial  for  test  planning,  rehearsal,  and  execution,  and  can  result  in  valid  data  being 
collected.  During  Phase  2,  they  also  identified  critical  constraints,  concerns,  and  methodologies 
associated  with  using  ADS  for  test  and  evaluation.  Finally,  the  ETE  Test  team  developed  and 
assessed  test  control  and  data  collection  methodologies  useful  for  ADS  testing. 
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6.0  Lessons  Learned 


6.1  Technical 

Testers  should  carefully  plan  the  development  of  the  simulations  and  links  comprising  their  ADS 
environment.  During  test  execution,  they  must  ensure  that  the  time  sources  are  synchronized  and 
continuously  monitor  PDU  traffic.  The  distributed  nature  of  ADS  testing  necessitates  special 
equipment  for  network  check-out  and  verification  and  requires  strict  configuration  control  of 
analysis  tools  and  collected  data. 

6.2  Infrastructure  and  Process 

ADS  test  planning  should  be  detailed  enough  to  encompass  key  requirements  at  the  earliest 
possible  stages,  yet  flexible  enough  to  accommodate  unexpected  situations  during  test  execution. 

A  conservative  development  approach  is  recommended  —  accomplish  risk  reduction  activities 
before  each  ADS  test  and  let  each  ADS  test  build  on  the  success  of  earlier  experiments. 
Successful  test  execution  requires  effective  internode  communication,  test  and  resource  control, 
and  data  management  procedures. 

7.0  Conclusions 
7.1  Utility 

An  ADS  environment  can  enhance  C4ISR  system  DT&E  and  OT&E.  In  comparison  with 
conventional  tests,  ADS  allows  testers  to  examine  C4ISR  systems  under  realistic  conditions  for 
longer  periods  of  time,  over  far  larger  battlespaces,  and  at  a  much  lower  cost.  This  versatile 
technology  can  provide  test  environments  that  include  large  numbers  of  entities,  entities  operating 
under  realistic  but  unsafe  conditions,  and  joint  and  combined  operations.  ADS  provides  C4ISR 
system  testers  with  greater  flexibility  in  designing,  executing,  and  analyzing  their  tests.  During 
DT&E,  ADS  allows  for  more  realistic  compliance  testing  of  C4ISR  subsystems  and  efficient 
implementation  of  the  test-fix-verify  cycle  for  software  development. 

ADS  testing  provides  dramatic  abilities  to  test  C4ISR  systems  or  components  in  that  system.  For 
instance,  in  the  ETE  Test  ADS  environment,  a  developmental  test  could  be  performed  on  the 
Joint  STARS  operations  and  control  subsystem  using  VSTARS  as  a  stimulus.  With  a  similar 
configuration,  an  operational  test  could  be  accomplished  on  a  LGSM.  The  ETE  Test  ADS 
environment  also  provides  arq)le  opportunities  to  install  new  components  for  various  types  of 
testing.  Links  to  airborne  weapon  system  simulators,  complimentary  sensor  feeds  or  other 
command  and  control  structures  can  be  easily  accomplished.  The  development  of  an  ADS  test 
environment  during  system  development  greatly  improves  opportunities  for  C4ISR  system 
training  after  the  completion  of  the  test.  The  same  infrastructure  developed  for  testing  can  be 
modified  and  transitioned  to  a  training  environment  resulting  in  program  savings.  This  technology 
allows  C4ISR  system  operators  to  confirm  current  tactics,  try  “what-if  ’  scenarios  and  new  tactics. 
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test  the  interoperability  and  compatibility  of  their  equipment,  and  gain  useful  experience  in  a 
realistic  operating  environment  containing  multiple  assets. 

7.2  Technical 

The  Phase  2  test  required  only  a  small  part  of  the  available  bandwidth  and  exhibited  a  low  PDU 
latency  rate  comparable  with  earlier  tests.  The  ETE  Test  network  was  highly  reliable  during 
Phase  2  testing  due  largely  to  the  ETE  Test  team’s  extensive  pretest  risk  reduction  efforts. 

7.3  Infrastructure 

Compared  to  conventional  testing,  ADS  testing  reduces  the  need  for  large  numbers  of  fielded 
personnel  and  vehicles.  The  ability  to  automatically  collect  and  analyze  test  data  also  reduces  the 
number  of  people  required  for  setup,  execution,  and  analysis.  ADS  test  success  relies  on  well- 
organized  test  control  and  data  management  procedures.  Distributed  testing  requires 
sophisticated  instrumentation,  trained  personnel  to  operate  and  maintain  that  equipment,  and 
funds  to  support  personnel  and  equipment  at  distant  test  nodes. 
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1.0  Introduction 


1.1  Joint  Advanced  Distributed  Simulation  Overview 

The  Joint  Advanced  Distributed  Simulation  (IADS)  Joint  Test  and  Evaluation  (JT&E)  was 
chartered  by  the  deputy  director,  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation),  Office  of  the  Under  Secretary  of  Defense  (OSD)  (Acquisition  and  Technology)  in 
October  1994  to  investigate  the  utility  of  advanced  distributed  simulation  (ADS)  technologies  for 
support  of  developmental  test  and  evaluation  (DT&E)  and  operational  test  and  evaluation 
(OT&E).  The  program  is  Ak  Force  led  with  Army  and  Navy  participation.  JADS  Joint  Test 
Force  (JTF)  manning  currently  includes  18  Air  Force  military,  4  Air  Force  civilians,  12  Army, 
and  1  Navy  civilian.  Science  Applications  International  Corporation  and  the  Georgia  Tech 
Research  Institute  provide  contracted  technical  support.  The  program  is  currently  scheduled  to 
end  in  March  2000. 

The  JADS  JT&E  charter  focuses  on  three  issues:  what  is  the  present  utility  of  ADS,  including 
distributed  interactive  simulation  (DIS),  for  test  and  evaluation  (T&E);  what  are  the  critical 
constraints,  concerns,  and  methodologies  when  using  ADS  for  T&E;  euid  what  are  the 
requirements  that  must  be  introduced  into  ADS  systems  if  they  are  to  support  a  more  complete 
T&E  capability  in  the  future.  From  these  issues,  objectives  and  measures  have  been  developed  to 
guide  the  evaluation. 

The  JADS  JTF  is  directly  investigating  ADS  applications  in  three  slices  of  the  T&E  spectrum;  the 
System  Integration  Test  (SIT)  which  explores  ADS  support  of  air-to-air  missile  testing;  the  End- 
to-End  (ETE)  Test  which  investigates  ADS  support  for  command,  control,  communications, 
computers,  and  intelligence,  surveillance  and  recoimaissance  (C4ISR)  testing;  and  the  Electronic 
Warfare  (EW)  Test  which  explores  ADS  support  for  EW  testing.  Each  test  will  apply  the  JADS 
objectives  emd  measures  as  appropriate  to  conduct  their  evaluation.  The  JTF  is  also  chartered  to 
observe  or  participate  at  a  modest  level  in  ADS  activities  sponsored  and  conducted  by  other 
agencies  in  an  effort  to  broaden  conclusions  developed  in  the  three  dedicated  test  areas. 

The  JADS  ETE  Test  is  the  subject  of  this  report  and  is  described  in  the  next  section;  the 
following  is  a  brief  synopsis  of  the  SIT  and  EW  Test. 

The  SIT  evaluated  the  utility  of  using  ADS  to  support  cost-effective  testing  of  an  integrated 
missile  weapon/launch  aircraft  system  in  an  operationally  realistic  scenario.  The  SIT  also 
evaluated  the  capability  of  the  JADS  Test  Control  and  Analysis  Center  (TCAC)  to  control  a 
distributed  test  of  this  type  and  to  remotely  monitor  and  analyze  test  results.  The  SIT  consisted 
of  two  phases  each  of  which  culminated  in  three  flight  missions.  The  missions  simulated  a  single 
shooter  aircraft  launching  an  air-to-air  missile  against  a  single  target  aircraft.  In  the  Linked 
Simulators  Phase  (LSP),  the  shooter,  target,  and  missile  were  all  represented  by  simulators.  In 
the  Live  Fly  Phase  (LFP),  the  shooter  and  target  were  represented  by  live  aircraft  and  the  missile 
by  a  simulator. 
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The  EW  Test  will  evaluate  the  utility  of  ADS  in  a  distributed  EW  environment.  The  first  phase 
was  open  air  testing  to  develop  a  performance  baseline  for  two  subsequent  test  phases.  The  first 
distributed  test  phase  employed  a  linked  architecture  using  Department  of  Defense’s  (DoD)  high 
level  architecture  (HLA)  which  included  a  digital  simulation  model  of  the  ALQ-131  self- 
protection  jammer,  threat  simulation  facilities,  and  constructive  models  which  support  replication 
of  the  open  air  environment.  In  the  second  phase,  an  installed  systems  test  facility  was  substituted 
for  the  digital  model.  In  both  distributed  test  architectures,  system  performance  data  wUl  be 
compared  with  live  fly  data  for  verification  and  validation  (V&V). 

1.2  Test  Overview 

The  ETE  Test  is  designed  to  evaluate  the  utility  of  ADS  to  support  testing  of  C4ISR  systems.  It 
win  conduct  its  T&E  utility  evaluation  in  an  ADS-enhanced  test  envirormient,  using  the  Joint 
Surveillance  Target  Attack  Radar  System  (Joint  STARS)  as  one  component  of  a  representative 
C4ISR  environment.  The  ETE  Test  will  also  evaluate  the  capability  of  the  JADS  TCAC  to 
control  a  distributed  test  of  this  type  and  to  remotely  monitor  and  analyze  test  results 

The  ETE  Test  is  using  distributed  simulation  to  assemble  an  enhanced  environment  for  testing 
C4ISR  systems.  The  intent  is  to  provide  a  complete,  robust  set  of  interfaces  from  sensor  to 
weapon  system,  including  the  additional  intermediate  nodes  that  would  be  found  in  a  tactical 
engagement.  The  test  will  trace  a  thread  of  the  complete  battlefield  process  from  target  detection 
to  target  assignment  and  engagement  at  corps  level  using  ADS.  It  will  allow  the  tester  to  evaluate 
the  thread  as  a  whole  or  the  contribution  of  any  of  the  parts  individually  and  to  evaluate  what 
effects  an  operationally  realistic  environment  has  on  the  system  under  test. 

The  ETE  Test  is  designed  to  add  additional  entities  in  a  seamless  manner  to  the  battlefield  seen  by 
Joint  STARS.  In  addition,  adding  some  of  the  complementary  suite  of  other  command,  control, 
communications,  computers  and  intelligence  (C4I)  and  weapon  systems  with  which  Joint  STARS 
would  interact  will  enable  the  test  team  to  evaluate  the  utility  of  an  ADS-enhanced  test 
environment. 

The  test  concept  (Figure  1)  is  to  use  ADS  to  supplement  the  operational  environment  experienced 
by  the  E-8C  and  light  ground  station  module  (LGSM)  operators.  By  mixing  available  live  targets 
with  targets  generated  by  a  constructive  model,  a  battle  array  approximating  the  major  systems 
present  in  a  notional  corps  area  of  interest  can  be  presented.  By  constructing  a  network  with 
nodes  representing  appropriate  C4I  and  weapon  systems,  a  more  robust  cross  section  of  players  is 
available  for  interaction  with  the  E-8C  and  LGSM  operators. 
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AFATDS  =  Advanced  Field  Artillery  Tactical  Data  System  ATACMS  =  Army  Tactical  Missile  System 

Bn  =  battalion  FDC  =  fire  direction  center 

Janus  =  interactive,  computer-based  simulation  of  combat  operations 

SCDL  =  surveillance  control  data  link  TAC  =  target  analysis  cell 

VSTARS  =  Virtual  Surveillance  Target  Attack  Radar  System 


Figure  1.  EXE  Test  Conceptual  Model 

Several  components  were  required  to  create  the  ADS-enhanced  operational  environment  used  in 
the  ETE  Test.  In  addition  to  Joint  STARS,  the  ETE  Test  required  a  validated  simulation  capable 
of  generating  entities  representing  the  rear  elements  of  a  threat  force.  As  discussed  in  Section 
1.3.1,  the  ETE  Test  team  selected  the  Janus  simulation  for  this  requirement.  Also,  simulations  of 
the  Joint  STARS  moving  target  indicator  (MTI)  radar  and  synthetic  aperture  radar  (SAR)  were 
needed  to  insert  the  simulated  entities  into  the  radar  stream  aboard  the  E-8C  while  it  was  flying  a 
live  mission.  Other  capabilities  used  to  support  the  test  include  simulations  or  subsets  of  the 
Army’s  artillery  command  and  control  process  and  a  simulation  of  the  Army  Tactical  Missile 
System  (ATACMS).  Communications  among  these  simulations  are  accomplished  using  such 
doctrinaUy  correct  means  as  the  CGS-100,  a  subsystem  of  the  Compartmented  All  Source 
Analysis  System  (ASAS)  Message  Processing  System  (CAMPS),  remote  workstations  (RWSs), 
and  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS)  message  traffic. 

The  ETE  Test  consists  of  four  phases.  Phase  1  developed  or  modified  the  components  that 
allowed  the  irdx  of  live  and  simulated  targets  at  an  E-8C  operator’s  console  and  LGSM  operator’s 
console.  Phase  2  evaluated  the  utility  of  ADS  to  support  DT&E  and  early  OT&E  of  a  C4ISR 
system  in  a  laboratory  environment.  Phase  3  transitions  portions  of  the  architecture  to  the  E-8C 
aircraft,  ensures  that  the  components  function  properly,  and  checks  that  the  synthetic  environment 
properly  interacts  with  the  aircraft  and  the  actual  LGSM.  Phase  4  will  evaluate  the  ability  to 
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perform  test  and  evaluation  of  the  E-8C  and  LGSM  in  a  synthetically  enhanced  operational 
environment  using  typical  operators. 

1.3  Phase  1  Overview 

This  section  summarizes  Phase  1  test  activities  which  were  completed  in  February  1998.  During 
Phase  1,  software  and  hardware  needed  to  establish  the  ETE  Test  ADS  environment  were 
developed,  modified,  and  integrated.  In  addition.  Phases  2  through  4  were  planned. 

The  ETE  Test  ADS  environment  components  developed  during  Phase  1  included  a  constructive 
simulation  to  provide  virtual  targets,  an  E-8C  simulation  called  the  Virtual  Surveillance  Target 
Attack  Radar  System  (VSTARS),  an  interface  to  allow  surveillance  control  data  link  (SCDL) 
traffic  from  VSTARS  to  be  displayed  in  the  LGSM,  and  an  ADS  network  suitable  for  integration 
and  testing. 

More  detailed  information  on  Phase  1  can  be  found  in  the  End-to-End  Interim  Report,  Phase  1, 
August  1998,  available  at  http://www.JADS.abq.com.  (After  1  March  2000  refer  requests  to  HQ 
AFOTEC/HO,  8500  Gibson  Blvd  SE,  Kirtland  Air  Force  Base,  New  Mexico  87117-5558,  or 
SAIC  Technical  Library,  2001  North  Beauregard  St.  Suite  80,  Alexandria,  Virginia  22311.) 

1.3.1  Phase  1  Approach 

The  JADS  ETE  Test  team  developed  requirements  for  a  constructive  simulation  and  then 
evaluated  available  simulations  against  these  requirements.  The  Janus  simulation,  developed  and 
managed  by  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 
(TRAC),  White  Sands  Missile  Range  (WSMR),  New  Mexico,  was  selected  as  the  simulation  best 
able  to  be  modified  to  meet  JADS’  requirements.  TRAC-WSMR  expanded  the  Janus  scenario 
driver  into  Janus  6.88D,  a  constructive  simulation  capable  of  supporting  up  to  10,000  individual 
entities  with  a  distributed  interactive  simulation  (DIS)  interface  to  the  ETE  Test  environment. 

The  JADS  ETE  Test  team  investigated  existing  simulations  of  Joint  STARS  and  determined  that 
none  of  them  met  the  needed  fidelity  requirements.  Northrop  Grumman,  the  developer  of  the 
E-8C,  created  a  laboratory  emulation  of  the  E-8C  radar  subsystem  and  the  capability  to  integrate 
the  E-8C  into  a  sjmthetic  environment.  The  VSTARS  is  a  laboratory  emulation  of  the  E-8C  radar 
subsystem  and  other  aircraft  components  which  can  receive  S3mthetic  targets  from  a  DIS  network 
and  provide  the  stimulus  to  display  these  targets  on  the  Advanced  Technology  Work  Station 
(ATWS)  or  LGSM.  The  radar  processor  simulator  and  integrator  (RPSI)  and  the  air  network 
interface  unit  (ANIU)  are  parts  of  the  VSTARS  which  are  installed  on  the  aircraft.  The  RPSI 
receives  radar  service  requests  (RSRs)  from  either  an  operator  workstation  (OWS)  or  a  ground 
station  module  replica  (GSMR)  and  provides  radar  target  reports  (moving  target  indicator  and 
synthetic  aperture  radar  to  the  OWS  and  GSMR. 

Phase  1  also  included  the  development  of  a  near  real-time  simulation  of  the  E-8C  synthetic 
aperture  radar.  The  JADS  ETE  Test  team,  through  the  Advanced  Research  Projects  Agency 
(ARPA)  War  Breaker  project,  conducted  a  trade  study  of  various  existing  simulations.  The 
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XPATCHES  simulation,  developed  by  Wright  Laboratory  (Dayton,  Ohio)  and  Loral  Defense 
Systems  (Goodyear,  Arizona),  was  selected  as  the  best  starting  point  for  the  E-8C  SAR 
simulation.  Lockheed  Martin  Tactical  Defense  Systems,  Goodyear,  Arizona,  developed  a  SAR 
simulation  emulating  the  Joint  STARS  SAR  operation.  This  simulation  system,  referred  to  as  the 
Advanced  Radar  Imaging  Emulation  System  (ARIES),  is  operationally  embedded  into  Northrop 
Grumman’ s  radar  processor  simulation  and  integrator. 

The  normal  connection  between  the  E-8C  and  its  associated  LGSM  is  through  a  line-of-sight  data 
link  called  the  surveillance  control  data  link  (SCDL).  After  considerable  investigation,  the  JADS 
ETE  Test  team  determined  that  this  link  could  not  be  easily  transmitted  via  commercial 
communications  lines.  Based  on  discussions  among  the  team,  Northrop  Grumman,  and  Motorola, 
the  JADS  ETE  Test  team  decided  to  develop  interfaces  which  transferred  the  normal  message 
traffic  between  the  E-8C  and  LGSM  rather  than  attempting  to  directly  transfer  the  SCDL 
messages.  Northrop  Grumman  and  Motorola  developed  an  interface  control  document  (ICD) 
which  defined  this  message  traffic.  Northrop  Grumman  included  the  capability  in  VSTARS  to 
capture  these  messages  and  divert  them  to  an  Ethernet;  Motorola  developed  an  interface  unit 
between  the  LGSM  and  an  Ethernet.  This  interface  unit  links  the  Ethernet  with  the  internal  1553 
databus  of  the  LGSM.  Additionally,  the  interface  unit  simulates  the  operation  of  the  ground  data 
terminal  requiring  the  LGSM  operator  to  perform  the  normal  linking  process  prior  to  receiving 
the  message  traffic  from  VSTARS. 

The  Phase  1  network  initially  connected  TRAC-WSMR  with  the  JADS  TCAC  in  Albuquerque, 
New  Mexico,  and  was  then  extended  from  the  TCAC  to  the  Northrop  Grumman  laboratory 
facilities  in  Melbourne,  Horida.  Late  in  Phase  1,  this  network  grew  to  include  links  from  the 
TCAC  to  Fort  Hood,  Texas,  and  Fort  Sill,  Oklahoma,  and  a  link  between  Northrop  Grumman  and 
Fort  Hood. 

1.3.2  Phase  1  Results 

Phase  1  identified  constraints  associated  with  ADS  testing.  One  key  constraint  was  the  ability  of 
the  DoD  infrastructure  to  support  ADS  test  and  evaluation.  A  measure  of  this  constraint  is  found 
in  the  amount  of  development  required  to  establish  a  synthetic  environment  with  which  to  conduct 
testing.  Phase  1  provided  insight  onto  the  development  required  to  support  a  test  of  this  type. 
Phase  1  also  demonstrated  the  application  of  a  systems  engineering  methodology  to  identify  the 
requirements  for  ADS  components,  evaluated  the  availability  of  ADS  components,  and  modified 
or  developed  the  components  to  meet  the  requirements. 

Extensive  testing  was  accompUshed  to  establish  and  verify  the  network  configuration.  (See 
Figures  4  and  5  for  the  final  configuration.)  Different  hardware  and  software  configurations  were 
tested.  The  use  of  user  data  protocol  (UDP)  packets  and  their  impact  on  the  ETE  Test 
environment  was  tested.  It  was  determined  that  UDP  imposed  some  restrictions  on  network 
speed.  Using  UDP  the  routers  were  only  capable  of  a  throughput  of  400  packets  per  second. 
UDP  is  used  among  aU  nodes  with  the  exception  of  the  SCDL  and  the  Advanced  Field  Artillery 
Tactical  Data  System  (AFATDS)  to  AFATDS  connection. 
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Data  management  and  analysis  methods  were  examined.  The  use  of  data  loggers  was  examined 
and  various  loggers  were  tested.  One  of  these  was  developed  in  support  of  Janus;  the  other  was  a 
product  from  the  U.S.  Army  Simulation,  Training,  and  Instrumentation  Command  (STRICOM). 
Neither  the  STRICOM  or  Janus  logger  provided  a  high  enough  time-stamp  accuracy,  kept 
processor  loads  minimized,  or  provided  flexible  enough  playback  accuracy  for  the  data  analysis 
needs  of  the  ETE  Test.  This  led  to  the  development  of  the  JADS  logger  and  the  JADS  player. 

The  JADS  logger  was  designed  to  place  a  priority  on  time  stamping  of  protocol  data  units 
(PDUs).  As  PDUs  are  received  they  are  time  stamped.  When  the  processor  has  time  it  logs  the 
PDUs.  It  also  has  a  very  minimal  user  interface  that  forgoes  graphics  in  favor  of  processor 
availabihty. 

The  JADS  player  is  able  to  operate  in  two  modes.  Using  the  first  mode,  an  operator  specifies  the 
playback  speed  multipher  that  controls  the  speed  of  the  PDU  playback.  The  second  mode 
specifies  the  rate  at  which  the  PDUs  will  be  played  back.  The  player  can  also  start  playback  at 
any  log  time. 

Synthetic  environment  analysis  data  tools  were  developed  to  aid  in  the  analysis  of  PDU  test  data. 
They  were  all  incorporated  into  a  single  tool  called  the  JADS  toolbox.  Some  of  the  features 
include  PDU  analysis  in  near  real  time,  predefined  analyses  and  outputs,  conversion  of  binary 
logfile  data  to  text  data,  PDU  replay,  and  conversion  utilities.  The  JADS  logger  was  used 
extensively  for  the  analysis  of  test  data. 

Data  management  techniques  explored  during  Phase  1  will  be  expanded  upon  during  the 
remainder  of  the  ETE  Test  program.  It  was  proven  that  data  collection  of  large  data  sets  and 
analysis  of  the  data  are  possible  and  even  facilitated  by  the  ETE  Test  environment. 

Phase  1  concluded  with  the  creation  of  a  large  ADS  synthetic  environment  for  C4ISR  testing.  All 
the  simulators  were  developed  and  then  connected  to  create  the  ETE  Test  C4ISR  testing 
environment.  The  environment  provided  the  capability  to  test  multiple  systems  simultaneously 
and  to  determine  impacts  of  one  system  upon  another  system.  Phase  2  took  the  environment  to 
the  next  step  and  actually  used  it  to  conduct  a  C4ISR  test. 
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2.0  Phase  2  Overview 


2.1  Phase  2  Purpose 

Phase  2  of  the  ETE  Test  was  designed  to  determine  the  utility  of  ADS  to  support  DT&E  and 
early  OT&E  of  a  C4ISR  system  in  a  laboratory-based  environment. 

2.2  Phase  2  Approach 

In  Phase  2,  the  E-8C  aircraft  was  represented  by  ground-based  simulations.  The  Janus  model 
simulated  the  threat  battlefield  entities.  VSTARS  emulated  the  radar  subsystem  of  the  E-8C 
aircraft  and  provided  the  inputs  needed  to  drive  target  displays  on  the  ATWS  (simulating  displays 
onboard  the  E-8C  aircraft)  and  the  LGSM.  The  resulting  radar  reports  were  provided  to  an 
actual  target  analysis  cell  (TAG)  for  analysis  and  target  assignment.  Fire  support  missions  were 
tasked  using  actual  AFATDS  messages  sent  to  the  Tactical  Army  Fire  Support  Model  (TAFSM) 
which  simulated  the  launch,  flyout,  and  detonation  of  an  ATACMS  missile. 

Figure  2  shows  the  organizational  structure  for  reporting  and  coordination  during  Phase  2  of  the 
ETE  Test. 


Bde  =  brigade  Bn  =  battalion  Co  =  company 

D&SA  =  Depth  and  Simultaneous  Attack  DDSA  =  deputy  director,  system  assessment  ID  =  infantry  division 

JPO  =  joint  program  office  MI  =  Military  Intelligence  PM  =  program  manager 

TEXCOM  =  U.S.  Army  Test  and  Experimentation  Command 

Figure  2.  ETE  Test  Organizational  Structure 


During  ETE  testing,  the  roles  and  responsibilities  of  these  organizations  are  as  follows. 

DDSA 
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The  deputy  director,  system  assessment  (DDSA)  in  Washington,  District  of  Columbia: 

•  Oversees  the  JADS  Joint  Test  and  Evaluation  (JT&E) 

•  Approves  JADS  financial  requirements 

•  Approves  the  program  test  plan  (FTP) 

•  Oversees  the  analysis  and  reporting  of  test  results 

JADS  JTF 

The  JADS  JTF  in  Albuquerque,  New  Mexico: 

•  Conducts  overall  planning,  execution,  analysis,  and  reporting  of  the  test 

•  Manages  funding  to  accomplish  the  test 

•  Develops  and  evaluates  JADS  issues,  objectives,  measures,  and  related  data  elements 

•  Develops  and  integrates  the  components  of  the  ETE  Test  ADS  environment 

•  Establishes  necessary  communication  links  with  test  participants 

•  Operates  the  Test  Control  and  Analysis  Center  during  tests 

•  Works  with  other  organizations  in  analyzing  test  data 

•  Reports  interim  and  final  results  to  OSD 

TRAC-WSMR 

TRAC-WSMR,  New  Mexico: 

•  Develops,  tests,  and  documents  Janus  6.88D  (an  expanded  variant  of  Janus)  for  JADS 

•  Assists  in  integrating  Janus  6.88D  into  the  ETE  Test  ADS  environment 

•  Assists  in  database  conversions 

•  Assists  in  developing  vignettes 

•  Assists  in  verification,  validation,  and  accreditation  (VV&A)  activities 

•  Assists  in  ETE  Test  execution 

TEXCOM  Lab 

The  U.S.  Army  Test  and  Experimentation  Command  (TEXCOM)  Lab  at  Fort  Hood,  Texas: 

•  Assists  in  scenario  and  vignette  development 

•  Assists  in  ETE  Test  execution 
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D&SA  Battle  Lab 


The  Depth  and  Simultaneous  Attack  (D&SA)  Battle  Lab  at  Fort  Sill,  Oklahoma: 

•  Provides  and  operates  the  TAFSM  and  AFATDS 

•  Assists  in  the  integration  of  the  ETE  Test  ADS  environment 

•  Assists  in  VV&A  activities  and  ETE  Test  execution 

U.S.  Army  III  Corps 

III  Corps  Fleadquarters  at  Fort  Hood,  Texas: 

•  B  Company  (Co),  303d  Military  Intelligence  (MI)  Battalion  (Bn),  504th  MI  Brigade  (Bde) 
supports  the  conduct  of  ETE  Test  Phase  2  and  Phase  4  events  with  LGSM(s)  and  a  target 
analysis  cell  (TAC)  and  assists  in  the  integration  of  the  ETE  Test  ADS  environment 

•  504  MI  Bde  provides  a  test  environment  for  the  ETE  Test  Phases  2  and  4 

•  Fire  Support  4th  Infantry  Division  (4  ID)  provides  an  AFATDS  and  personnel  to  support  the 
ETE  Test  Phases  2  and  4 

Joint  STARS  Joint  Program  Office  (JPO) 

Joint  STARS  JPO,  Hanscom  Air  Force  Base  (AFB),  Massachusetts,  provides  access  to  the  Joint 
STARS  JTF  and  Northrop  Grumman. 

The  Joint  STARS  JTF  of  the  Joint  STARS  JPO  in  Melbourne,  Florida: 

•  Supports  conduct  of  testing  in  all  phases 

•  Analyzes  Joint  STARS  test  results  and  provides  evaluations  according  to  JADS  objectives 

•  Assists  in  VV&A  activities 

Northrop  Grumman  Aerospace  Corporation 

Northrop  Grumman,  Electronics  and  Systems  Integration  Division  in  Melbourne,  Florida: 

•  Designed,  developed  and  integrated  the  radar  processor  simulation  and  integrator  (RPSI) 

•  Developed  the  Virtual  Surveillance  Target  Attack  Radar  System  (VSTARS) 

•  Conducts  and  assists  in  verification  and  validation  (V&V)  activities 

•  Assists  in  E-8C  mission  planning 

•  Operates  VSTARS  during  ETE  Test  phases 

Contracting  with  Northrop  Grumman  is  conducted  through  Rome  Laboratory  in  New  York. 


17 


93d  Wing 


93d  Wing  at  Robins  AFB,  Georgia: 

•  Provided  operators  during  Phase  2  of  the  ETE  Test. 

2.3  Test  Objectives 

The  JADS  issues,  test  objectives,  and  subobjectives  for  Phase  2  are  described  below.  Each 
subobjective  in  turn  encompassed  one  or  more  test  measures.  In  Section  4  these  issues, 
objectives,  subobjectives,  and  test  measures  are  discussed  in  terms  of  their  intent,  the  associated 
data  collection  methodology,  and  operational  test  results. 

JADS  Issue  1.  What  is  the  present  utility  of  ADS,  including  DIS,  for  T&E? 

JADS  Objective  1-1.  Assess  the  validity  of  data  from  tests  utilizing  ADS,  including  DIS, 
during  test  execution. 

JADS  Objective  1-2.  Assess  the  benefits  of  using  ADS,  including  DIS,  in  T&E.  This  test 
objective  was  broken  down  into  subobjectives.  (Subobjective  1-2-1  is  not  applicable  to  the 
ETE  Test  Phase  2.) 

JADS  Subobjective  1-2-2.  Assess  ADS  capability  to  support  T&E  planning  and  test 
rehearsal. 

JADS  Subobjective  1-2-3.  Assess  ADS  capability  to  support  T&E  shortfalls. 

JADS  Issue  2.  What  are  the  critical  constraints,  concerns,  and  methodologies  when  using  ADS 
for  T&E? 

JADS  Objective  2-1.  Assess  the  critical  constraints  and  concerns  in  ADS  performance  for 
T&E.  This  objective  was  broken  down  into  subobjectives.  (Subobjective  2-1-1  is  not 
applicable  to  the  ETE  Test  Phase  2.) 

JADS  Subobjective  2-1-2.  Assess  network  and  communications  performance  constraints 
and  limitations. 

JADS  Subobjective  2-1-3.  Assess  the  impact  of  ADS  reliability,  availability,  and 
maintainability  on  T&E. 

JADS  Objective  2-2.  Assess  the  critical  constraints  and  concerns  in  ADS  support  systems 
for  T&E.  This  objective  was  broken  down  into  subobjectives. 
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JADS  Subobjective  2-2-1.  Assess  the  critical  constraints  and  concerns  regarding  ADS 
data  management  and  analysis  systems. 

JADS  Subobjective  2-2-2.  Assess  the  critical  constraints  and  concerns  regarding 
configuration  management  of  ADS  test  assets. 

JADS  Objective  2-3.  Develop  and  assess  methodologies  associated  with  ADS  for  T&E. 
(Subobjective  2-3-1  is  not  applicable  to  the  ETE  Test  Phase  2.) 

JADS  Subobjective  2-3-2.  Develop  and  assess  methodologies  associated  with  test 
execution  and  control  for  tests  using  ADS. 

2.4  Phase  2  Methodology 

2.4.1  Tactical  Vignettes 

The  tactical  vignettes  for  the  ETE  Test  activities  are  unclassified.  The  ETE  Test  team  enhanced  a 
TRADOC-approved,  54-hour  corps  battlefield  simulation  (CBS)  scenario  by  repheating  an  Iraqi 
corps  rear  area  of  operations  in  Iraq.  Table  1  describes  the  five  tactical  vignettes  created  in  Janus 
6.88D;  each  vignette  is  six  hours.  The  following  targets  were  present  in  the  150  x  150  kilometer 
(km)  Southwest  Asia  (SWA)  terrain  box:  air  defense  artillery  (ADA)  sites,  command  and  control 
sites,  lines  of  communications  (convoys),  logistics  bases,  and  concentrations  of  armor  and  artillery 
units. 


Table  1.  Vignettes  Used  During  ETE  Testing 


Vignette 

Description 

Number  of 
Entities 

1 

Prehostility  phase 

9,897 

2 

Preemptive  strikes 

9,757 

3 

Hammurabi  Division  logistical  operations 

9,904 

4 

Commitment  of  the  Hammurabi  Division 

9,781 

5 

General  headquarters  (GHQ)  depots  to 
corps  and  divisional  logistical  operations 

9,950 

2.4.2  Test  Configuration 

2.4.2. 1  Phase  2  Synthetic  Environment 

Several  components  were  required  to  create  the  ADS-enhanced  environment  used  in  Phase  2. 
Figure  3  provides  an  overview  of  the  Phase  2  synthetic  environment. 
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The  ETE  Test  used  the  Janus  6.88D  simulation  to  generate  the  entities  representing  the  elements 
in  the  rear  of  a  threat  force.  Janus  generated  entity  state  PDUs  (ESPDUs)  for  the  threat  force 
which  were  transmitted  to  the  E-8C  simulation  via  the  Test  Control  and  Analysis  Center  (TCAC). 
TRAC-WSMR  provided  the  Janus  scenario  feed. 

The  TCAC  in  Albuquerque,  New  Mexico,  provided  test  control.  The  JADS  Network  and 
Engineering  (N&E)  team  monitored  the  health  of  the  ETE  Test  network  and  ensured  that 
adequate  data  flowed  in  support  of  the  test. 


TCAC 

Albuquerque,  New  Mexico 


SCDL 


Grumman  Aerospace  Labs 
E-8C  Simulation 

Melbourne,  Florida 


T-1  =  digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1,544  megabits  per  second 


Figure  3.  ETE  Test  Phase  2  Synthetic  Environment 

The  Joint  STARS  E-8C  simulation,  VSTARS,  represents  the  radar  subsystem  of  the  Joint  STARS 
E-8C  in  a  laboratory  environment.  It  is  composed  of  a  distributed  interactive  simulation  network 
interface  unit  (NIU),  a  radar  processor  simulator  and  integrator  (RPSI)  that  contains  the  two  real¬ 
time  radar  simulations  with  necessary  databases,  and  various  simulations  of  E-8C  processes. 
Figure  4  provides  more  information  on  the  VSTARS  architecture.  VSTARS  was  operated  at  the 
Northrop  Gmmman  Surveillance  and  Battle  Management  Systems  facility  in  Melbourne,  Florida. 

The  TAC,  fire  support  (provided  by  the  AFATDS),  and  a  LGSM  were  stationed  at  Fort  Hood, 
Texas. 

Communications  among  these  C4I  systems  employed  such  doctrinally  correct  means  as  the  CGS- 
100,  a  subsystem  of  the  CAMPS,  remote  workstations,  and  AFATDS  message  traffic.  The 
AFATDS  messages  were  transmitted  between  the  AFATDS  located  at  Fort  Hood  and  the 
AFATDS  located  at  Fort  Sih  using  actual  tactical  protocols,  rather  than  DIS  PDUs.  Also,  the 
SCDL  messages  were  transmitted  between  VSTARS  and  the  LGSM  using  a  dedicated  link,  a 
special-purpose  interface,  and  the  actual  tactical  protocols. 
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SMO 


Network 
- i 

ESPDUs 


CDP  -  central  data  processor  SM&C  -  system  management  and  control 

Mc&IS  -  management  and  integration  software  SMO  -  system  management  officer 


Figure  4.  VSTARS  Architecture 


The  TAFSM  simulation  at  Fort  SiU  modeled  the  ATACMS  battalion  and  sent  the  fire  and 
detonate  PDUs  to  the  Janus  6.88D  simulation.  In  turn,  Janus  modeled  the  engagement  results  and 
reflected  the  results  in  the  synthetic  environment. 
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2.4.2.2  Phase  2  Network 


Figure  5  provides  a  more  detailed  description  of  the  EXE  Test  network. 


Figure  5.  EXE  Test  Phase  2  Network  Diagram 


2.4.2. 3  Test  Control  and  Monitoring 

During  Phase  2,  EXE  Test  and  Network  and  Engineering  team  members  performed  test  control 
from  the  XCAC.  Test  control  consisted  of  three  major  areas  —  network  monitoring, 
communications,  and  test  procedures. 

The  Network  and  Engineering  team  conducted  network  monitoring  in  the  XCAC  using  hardware 
and  software  tools.  The  software  consisted  of  commercial  products  and  test-specific  tools 
developed  by  JADS  analyst/programmers.  They  used  the  following  systems. 

Silicon  Graphics,  Inc.  (SGI)  Indy  -  JADS  logger 

SGI  Indy  -  time  server 

SGI  Indigo  -  NEXVisualyzer™ 

SUN  SPARC  5  -  Spectrum'^^ 

Line  printers 
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JADS  analyst/programmers  developed  the  JADS  logger.  This  software  recorded  aU  PDU  traffic 
at  individual  sites.  AU  nodes,  with  the  exception  of  Fort  Hood,  had  a  JADS  logger  instaUed.  The 
logger  recorded  the  receipt  of  the  PDU  and  time  stamped  it  using  an  accurate  time  source.  These 
data  were  used  to  analyze  PDU  transmission  performance  over  the  network. 

JADS  analyst/programmers  also  developed  a  time  server  which  provided  the  accurate  time  source 
needed  for  the  JADS  logger.  The  time  server  was  tied  to  a  global  positioning  system  (GPS) 
receiver  located  in  the  TCAC  and  provided  time  to  aU  nodes  with  an  accuracy  of  100 
microseconds.  The  software  also  contained  monitoring  tools  to  track  the  time  servers 
performance  over  an  8-hour  test  period. 

Cabletron  Spectrum™,  NETVisuaUzer™,  and  line  printers  were  used  to  provide  network 
monitoring.  Spectrum™  measured  bandwidth  utilization.  This  tool  recorded  the  percentage  of 
bandwidth  used,  as  weU  as  bandwidth  loading  on  a  network  segment.  NETVisuaUzer™  software 
displayed  real-time  bandwidth  use  in  a  roUing  bar  graph  format  for  quick  visual  reference.  The 
Une  printers  provided  a  printout  of  network  router  status.  Any  faUure  or  high  bit  error  rate 
resulted  in  a  printout  showing  the  problem  and  identity  of  the  offending  router. 

Communication  among  the  distributed  ETE  Test  nodes  was  critical.  The  TCAC  provided  aU 
needed  communication  systems.  Dedicated  phone  circuits,  residing  on  the  T-1  Unes,  provided 
classified  and  unclassified  service.  The  unclassified  Une  aUowed  connectivity  among  the  TCAC, 
Fort  Hood,  Fort  SiU,  and  TRAC-WSMR.  The  classified  Une  aUowed  connectivity  among  the 
TCAC,  Fort  Hood,  and  Northrop  Grumman.  These  lines  aUowed  the  operator  to  select  the 
desired  site,  lift  the  receiver,  and  connect  directly  to  that  site.  The  TCAC  could  select  multiple 
sites  for  conference  calls  on  these  Unes.  In  addition  to  these  lines,  the  ETE  Test  team  used  an 
unclassified  conference  line  to  coordinate  such  test  events  as  network  checks  and  after-action 
debriefs.  This  line  allowed  up  to  ten  participants  to  connect  at  one  time. 

Test  procedures  were  required  to  provide  effective  control  of  aU  test  nodes  during  test  events. 
The  test  procedures  were  in  checkUst  format,  which  provided  for  standardization  among  the 
distributed  nodes.  The  network  checkUst  was  most  critical  and  was  used  to  initiaUze  the  network 
before  the  test.  Other  checkUsts  included  those  used  to  start  up  hardware  and  software  at 
individual  nodes,  as  weU  as  the  checkUst  used  by  the  TCAC  test  controUer  to  start  and  stop  the 
overall  test. 
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2.5  Planned  Phase  2  Schedule 


Figure  6  provides  a  schedule  of  the  top  level  tasks  for  Phase  2  of  the  ETE  Test. 


Task  Name 

FY96 

FY97 

FY98 

FY99  li 

Qtr  1  Qtr  2  Qtr  3  Qtr  4 

Network  Expansion 

Network  Activation 

Test/Benchmark/Modification 

VV&A 

FI  &  Risk  Reduction  Tests 

Operational  Test 

Report 

t 

t 

t _ t 

ttiit 

1 

t 

f 

^  1-  w  y\  Previous  scheduled  Previous  scheduled 

IT  Scheduled  completion  |  Actual  completion  completion  -  in  future  ▼  completion  -  date  passed 


FI  =  Functionality  and  Integration  Tests  #1-4 


Figure  6.  ETE  Test  Schedule 


2.6  Phase  2  Costs 

This  report  does  not  describe  the  costs  of  the  Phase  2  ETE  Test.  Rather,  the  report  on  Phase  4  of 
the  ETE  Test  will  include  a  Work  Breakdown  Structure  covering  the  costs  of  all  four  phases  of 
the  ETE  Test.  The  Phase  4  ETE  Test  report  will  be  published  in  summer,  1999. 
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3.0  Phase  2  Execution  Results 


Four  functionality  and  integration  (FI)  tests  and  one  risk  reduction  test  were  conducted  prior  to 
Phase  2  operational  testing. 

3.1  ETE  Test  Functionality  and  Integration  (FI)  Tests  #1  and  #2 

FI  tests  #1  and  #2  took  place  in  spring  1998.  These  two  tests  focused  on  establishing  the 
individual  nodes  and  links  comprising  the  ETE  Test  ADS  environment  and  did  not  involve  any 
data  collection. 

3.2  ETE  Test  FI  Tests  #3  and  #4  and  Risk  Reduction  Test 

For  all  three  tests,  the  ETE  Test  network  consisted  of  Janus  at  WSMR,  VSTARS  at  Melbourne, 
TAFSM  at  Fort  Sill  and  the  LGSM  at  Fort  Hood. 

-  During  each  test  entity  state  PDUs  (ESPDUs)  were  broadcast  from  WSMR  to  Northrop 
Grumman  via  the  TCAC.  In  addition,  entity  state,  fire,  detonate,  transmit,  and  signal  PDUs  were 
broadcast  from  Fort  Sill  to  WSMR  via  the  TCAC. 

-  The  Spectrum™  network  analysis  tool  measured  bandwidth  utilization  on  the  classified  links 
connecting  TCAC  to  Northrop  Grumman  and  Northrop  Grumman  to  Fort  Hood. 

-  Automated  data  were  collected  using  PDU  loggers  at  the  nodes.  Taking  advantage  of  the 
networked  nature  of  the  ADS  environment,  JADS  analysts  used  the  file  transfer  protocol  (FTP)  to 
send  the  data  collected  at  the  nodes  to  JADS.  These  data  were  then  compressed  and  converted 
for  analysis  on  JADS  UNIX^'^-based  analysis  tools.  Operational  data  were  collected  using  log 
sheets  at  each  node. 

-  Draft  test  procedures  were  documented  before  each  test.  During  the  test  readiness  review 
before  each  test,  the  baseline  configuration  was  verified.  During  each  test,  the  test  controller 
revised  and  augmented  aU  procedures  used  to  control  the  test  configuration,  environment  data, 
and  ADS  network. 

3.2.1  FI  Test  #3 

FI  test  #3  took  place  on  26-28  May  1998. 

-  Janus  vignettes  1  and  3  were  used  to  simulate  entities  and  movements  for  a  threat  corps  rear 
area  of  operations.  Vignette  1  added  9,897  entities  to  test  execution;  vignette  2  added  9,757. 

-  Testing  for  most  of  day  one  was  scrubbed  because  of  a  VSTARS  malfunction.  None  of  the 
nodes  experienced  any  problems  on  days  two  or  three. 
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-  The  network  was  unavailable  for  testing  for  38  minutes  out  of  a  scheduled  20  hours  and  33 
minutes. 

-  For  the  TCAC-Northrop  Grumman  elassified  link  the  packet  rate  averaged  17  packets/second, 
network  load  (bandwidth  utilized)  during  active  test  execution  averaged  1%,  and  peak  load  never 
exceeded  30%  of  link  capacity  during  the  three  days  of  testing.  Packet  rate  and  network  load  did 
increase  significantly  over  the  TCAC-Grumman  link  immediately  following  active  testing  each  day 
because  PDU  logger  data  files  were  being  imported  from  Northrop  Grunmian  to  the  TCAC. 

-  For  the  three-day  test,  total  PDU  losses  and  latency  were 


Node  A 

Node  B 

PDUs  Sent 

PDUs 

Received 

PDUs  Lost 
Total/% 

Min/Max/Mean 
Latency  (s) 

WSMR 

TCAC 

934,971 

872,297 

62,674/6.70% 

.026/.120/.029 

TCAC 

Grumman 

872,297 

67,730/7.76% 

0.031/.109/.034 

Fort  Sill 

WSMR 

2,543 

30/1.12% 

— 

3.2.2  FI  Test  #4 

FI  test  #4  took  place  on  25-26  June  1998. 

-  Janus  vignettes  1  and  2  were  used  to  simulate  entities  and  movements  for  a  threat  corps  rear 
area  of  operations.  Vignettes  1  and  2  each  added  9,766  entities  to  test  execution. 

-  Testing  on  25  June  was  scrubbed  for  less  than  1  hour  because  of  the  power  spike  which 
disrupted  ETE  testing  at  WSMR.  Both  the  Fort  Hood  and  Northrop  Grumman  nodes 
experienced  problems  on  26  June  with  either  a  router  or  the  Ethernet  interface  causing  the 
problem.  Fort  Sill  did  not  experience  any  problems  on  either  day. 

-  The  network  was  unavailable  for  1  hour  and  13  minutes  out  of  a  scheduled  12  hours  and  55 
minutes. 

-  Packet  rate  experienced  for  the  classified  links  averaged  around  20  packets/second.  Network 
load  (bandwidth  utilized)  during  active  test  execution  averaged  1%,  and  peak  load  never  exceeded 
5%  of  link  capacity  during  the  two  days  of  testing.  Packet  rate  and  network  load  did  increase 
significantly  over  the  TCAC-Grumman  link  immediately  following  active  testing  each  day  because 
PDU  logger  data  files  were  being  imported  from  Northrop  Grumman  to  the  TCAC. 

-  Total  PDU  losses  and  latency  were 


Node  A 

Node  B 

PDUs  Sent 

PDUs 

Received 

PDUs  Lost 
Total/% 

Min/Max/Mean 
Latency  (s) 

WSMR 

TCAC 

497,721 

473,569 

24,152/4.85% 

.017/.139/.034 

TCAC 

Grumman 

473,569 

15,654/3.31% 

.028/.060/.034 

Fort  Sill 

WSMR 

670 

624 

46/6.87% 

— 
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3.2.3  Risk  Reduction  Test 


The  risk  reduction  test  took  place  on  7-10  and  13  July  1998. 

-  Janus  vignettes  1  and  3  were  used  to  simulate  entities  and  movements  for  a  threat  corps  rear 
area  of  operations.  Vignette  1  added  9,897  entities  to  test  execution;  vignette  3  added  9,904 
entities. 

-  On  7  July,  Fort  Sill  experienced  a  minor  problem  because  of  a  trunk  card  outage.  Testing  was 
interrupted  for  approximately  1  hour  on  8  July  because  of  problems  with  Janus.  Janus  froze  up, 
was  restarted,  and  then  froze  again,  causing  the  test  to  stop  prematurely  for  that  day.  Both  Fort 
Hood  and  Northrop  Grumman  experienced  problems  on  9  July  because  of  difficulties  with  a 
router/Ethernet  interface.  Northrop  Grumman  also  experienced  a  minor  problem  on  13  July 
because  of  a  trunk  card  outage  at  JADS. 

-  The  network  was  unavailable  for  testing  for  2  hours  and  40  minutes  out  of  a  scheduled  35  hours 
and  7  minutes. 

-  For  the  TCAC-Northrop  Grumman  classified  link,  packet  rate  averaged  19  packets/second, 
network  load  (bandwidth  utilized)  during  active  test  execution  averaged  1%,  and  peak  load  never 
exceeded  4%  of  link  capacity  during  the  five  days  of  testing.  For  the  Northrop  Grumman  -  Fort 
Hood  classified  link,  packet  rate  averaged  34  packets/second,  network  load  averaged  1%,  and 
peak  load  never  exceeded  5%  of  link  capacity.  Packet  rate  and  network  load  did  increase 
significantly  over  the  TCAC-Grumman  link  immediately  following  active  testing  each  day  because 
PDU  logger  data  files  were  being  imported  from  Northrop  Grumman  to  the  TCAC. 

-  Total  PDU  losses  and  latency  were  as  follows: 


Node  A 

NodeB 

PDUs  Sent 

PDUs 

Received 

PDUs  Lost 
Total/% 

Min/Max/Mean 
Latency  (s) 

WSMR 

TCAC 

1,667,371 

1,667,283 

887.005% 

.007/.142/.041 

TCAC 

Grumman 

1,667,283 

1,604,069 

63,214/3.79% 

.031/.361/.036 

Fort  Sill 

WSMR 

3,386 

3,385 

l/.03% 

— 

3.3  Operational  Test 

The  operational  test  portion  of  the  Phase  2  test  took  place  from  14  September  through  7  October 
1998.  The  specific  measures  addressed  during  the  test  and  the  data  collected  in  support  of  those 
measures  are  discussed  in  Section  4. 
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4.0  Analysis  of  Test  Objectives 


During  the  operational  test  portion  of  Phase  2  of  the  ETE  Test,  JADS  analysts  collected 
information  to  address  the  issues,  JADS  objectives,  and  test  measures  as  outlined  in  the  JADS 
Program  Test  Plan  (PTP)  and  the  ETE  Test  Data  Management  and  Analysis  Plan  (DMAP).  Only 
those  subobjectives  and  measures  evaluated  using  Phase  2  results  are  discussed. 

4.1  JADS  Issue  1.  What  is  the  present  utility  of  ADS,  including  DIS,  for  T&E? 

This  objective  determines  the  extent  to  which  ADS  technology  can  support  the  T&E  of  current 
and  future  C4ISR  systems. 

4.1.1  JADS  Objective  1-1.  Assess  the  validity  of  data  from  tests  utilizing  ADS,  including 
DIS,  during  test  execution. 

During  Phase  2  of  the  ETE  Test,  the  ETE  Test  team  examined  the  validity  of  data  from  an  ADS 
configuration  incorporating  the  VSTARS  simulation,  Janus  simulation,  and  other  components  into 
a  C4ISR  architecture. 

JADS  Measure  l-l-O-l.  Degree  to  which  ADS  provides  valid  system  under  test  (SUT)  data. 

JADS  Measure  1-1-0-2.  Percentage  of  ADS  data  which  are  valid  (data  supporting  test 
measures  which  are  timely,  accurate,  reliable,  and  otherwise  faithfully  represent  real  world 
systems  data). 

These  two  test  questions  gauge  the  ability  of  an  ADS  environment  to  provide  valid  data  for  a 
C4ISR  system  under  test.  The  first  measure  addresses  the  validity  of  the  SUT  output  data  which 
form  the  data  elements  for  evaluating  SUT  measures.  The  second  measure  provides  an 
assessment  of  the  input  data  provided  to  the  SUT  by  the  ADS  environment. 

These  measures  were  primarily  addressed  through  implementation  of  the  Phase  2  V&V  plan. 
Since  JADS  is  a  DoD-sponsored  joint  test,  the  basis  for  the  Phase  2  V&V  was  the  DIS  nine-step 
VV&A  process  model  and  its  accompanying  Recommended  Practice  for  Distributed  Interactive 
Simulation  —  Verification,  Validation,  and  Accreditation  (draft-21  May  1996).  Figure?  provides 
an  overview  of  the  VV&A  process  model.  Note  that  this  model  has  been  extensively  discussed 
with  numerous  members  of  the  DIS  modeling  and  simulation  community,  has  been  generally 
accepted  by  V&V  practitioners,  and  is  currently  being  balloted  as  a  standard. 
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Figure  7.  DIS  Nine-Step  VV&A  Process  Model 

EXE  Test  VV&A  represented  a  tailoring  and  implementation  of  the  nine-step  process  to  a 
multiservice  test  of  a  major  system,  Joint  STARS,  augmented  with  ADS.  The  tailored  ETE  Test 
process  model  used  is  described  in  the  Phase  2  V&V  reports,  Phase  2  Verification  and  Validation 
Results  for  the  End-to-End  Test  and  Verification  and  Validation  Report  for  Phase  2  of  the 
Virtual  Surveillance  Target  Attack  Radar  System  (VSTARS).  (Available  from  IADS,  2050A  2nd 
Street  SE,  Kirtland  Air  Force  Base,  New  Mexico,  87117-5522.  After  1  March  2000  refer 
requests  to  HQ  AFOTEC/HO,  8500  Gibson  Blvd  SE,  Kirtland  Air  Force  Base,  New  Mexico 
87117-5558,  or  SAIC  Technical  Library,  2001  North  Beauregard  St.  Suite  80,  Alexandria, 
Virginia  22311.) 

The  results  from  implementing  the  ETE  Test  process  model  for  the  V&V  of  the  Phase  2  ADS 
configuration  are  detailed  in  the  Phase  2  V&V  reports  and  are  summarized  as  follows. 

-  Verification  of  Janus  6.88D.  The  following  were  verified. 

—  Janus  6.88D  was  capable  of  simulating  at  least  5000  entities  with  at  least  twenty-five 
percent  moving. 

—  Jcuius  6.88D  operators  were  capable  of  fuUy  interacting  with  a  scenario  while  it  was 
running.  In  particular,  the  Janus  site  representative  interacted  with  the  scenario  after 
several  ATACMS  impacts  by  altering  the  behavior  of  the  surviving  entities  within  the 
immediate  area  of  the  impact. 

—  Janus  6.88D  accepted  and  processed  scenarios  with  a  duration  longer  than  eight  hours. 

—  Janus  6.88D  was  capable  of  proceeding  at  a  pace  representative  of  near  real  time. 

—  Janus  6.88D  was  capable  of  utiMzing  scenarios  conducted  upon  terrain  representing  a 
simulation  area  of  at  least  170  km  by  170  km. 
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Verification  of  VSTARS.  The  following  were  verified. 

—  VSTARS  received  and  integrated  virtual  data  from  the  Phase  2  ADS  environment. 

—  VSTARS  operated  in  three  modes;  live  only,  mixed  live  and  virtual,  and  virtual  only  using 
the  standard  Joint  STARS  MTI  message  format. 

—  The  radar  timeline  was  not  impacted  by  the  MTI  simulation. 

—  VSTARS  processed  parameter  data  in  the  same  format  as  Joint  STARS. 

—  VSTARS  displayed  live  SARs  in  hve  areas  of  interest  and  virtual  SARs  in  both  mixed  and 
virtual  areas  using  the  standard  Joint  STARS  SAR  message  format. 

—  VSTARS  permitted  aU  of  the  installed  operator  workstation  software  to  function  without 
abnormal  fault  messages  occurring  with  minor  exceptions. 

Verification  of  compliance  standards  (DIS  step  2).  It  was  verified  that  the  PDUs  emitted  by 
each  simulation  adhered  to  the  prescribed  format. 

—  It  was  verified  that  Janus  6.88D  issued  DIS  2.0.4  entity  state  protocol  data  units 
(ESPDUs)  that  conformed  in  content  and  format  with  the  DIS  2.0.4  standards  as  amended 
by  JADS.  (JADS  modified  the  ESPDU  time-stamp  format  from  time  passed  since  the 
beginning  of  the  current  hour  to  milliseconds  since  the  beginning  of  the  vignette.  This 
allowed  testers  to  trace  an  ESPDU  back  to  a  discrete  event  that  occurred  within  the  Janus 
vignette.) 

“  Note  that  the  AFATDS  located  at  Fort  Hood  communicated  directly  with  the  AFATDS  at 
Fort  Sill  using  standard  AFATDS  message  traffic  instead  of  DIS  PDUs. 

Verification  of  compatibility  (DIS  step  6).  It  was  verified  that  the  modeling  and  simulation 
(M&S)  components  exchanged  data  and  interacted  appropriately  with  each  other;  that 
individual  components  correctly  used  the  common  data  (e.g.,  terrain,  weather)  to  generated 
their  portion  of  the  synthetic  environment,  and  that  the  overall  implementation  was  adequate 
to  address  the  exercise  requirements.  It  was  also  verified  that  the  network  allowed  that 
transfer  of  information  between  the  components  without  corruption  and  that  the  individual 
components  shared  a  common  perspective  of  the  virtual  reality  produced  by  the  exercise. 
Specific  considerations  were  as  follows. 

—  It  was  verified  that  Janus  6.88D  received  each  ESPDU,  fire  PDU,  and  detonate  PDU 
issued  to  it  by  the  network  and  took  the  appropriate  action  as  dictated  by  the  PDU.  In 
particular,  Janus  correctly  identified  ATACMS  launches  initiated  by  Fort  SOI  which  were 
outside  the  scenario  terrain  box. 

—  It  was  verified  that  TAFSM  accepted  artOlery  missions  using  AFATDS  messages  and  that 
TAFSM  issued  a  fire  and  detonate  PDU  whenever  an  ATACMS  missOe  was  fired  and 
subsequently  detonated. 

Validation  of  the  ETE  Test  Phase  2  synthetic  environment  (SE)  (DIS  Step  7).  It  was  ensured 
that  the  integrated  simulations  were  adequate  to  satisfy  the  ETE  Test  requirements  such  that 
the  behavior  and  performance  of  the  SE  mapped  sufficiently  to  real-world  counterparts  for  the 
specific  ETE  Test  application.  The  conclusion  that  the  ETE  Test  SE  was  a  realistic 
representation  of  the  real  world  was  based  on  structured  interviews  with  the  VSTARS, 
LGSM,  and  TAC  operator  subject  matter  experts  (SME)  participating  in  the  test. 
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—  It  was  specifically  validated  that  Janus  6.88D  represented  vehicle  behavior  to  the  degree 
detectable  by  the  Joint  STARS,  as  judged  by  Joint  STARS  operator  SMEs  viewing  vehicle 
movement  presented  by  the  Joint  STARS  operator  workstation.  Realistic  vehicle  behavior 
was  achieved  after  correcting  an  anomaly  observed  during  the  ETE  risk  reduction  test. 

—  Portions  of  convoys  missed  turns  and  wandered  off  into  the  desert  during  the  entire 
scenario.  The  lost  portion  of  the  convoy  would  then  jump  back  into  formation  after  a 
period  of  time  and  resume  normal  movement. 

—  It  was  determined  that  when  many  vehicles  were  moving  Janus  was  not  sending 
change  of  state  PDUs  at  a  high  enough  rate.  When  vehicles  are  moving,  ESPDUs 
must  be  sent  for  any  change  of  state  (starting,  stopping,  turning,  or  changing  speed 
beyond  preset  limits)  in  addition  to  the  normal  heartbeat  ESPDUs  for  stationary 
entities.  When  Janus  did  not  send  enough  change  of  state  ESPDUs,  VSTARS 
displayed  the  vehicle  motion  in  a  straight  line  from  the  last  update. 

—  The  solution  was  to  turn  off  the  heartbeat  prior  to  the  beginning  of  entity  movement  so 
that  Janus  issued  only  change  of  state  ESPDUs  as  the  changes  occurred.  The  rate  at 
which  Janus  sent  change  of  state  PDUs  was  also  dramatically  increased  from  a 
maximum  rate  of  15  PDUs  per  second  to  100  PDUs  per  second.  This  allowed 
VSTARS  to  more  accurately  display  the  convoys  movement  on  their  true  path. 

—  For  VSTARS  validation,  operators  with  Joint  STARS  experience  performed  a  two-hour 
mission  using  VSTARS  and  were  then  asked  to  compare  VSTARS  performance  with 
Joint  STARS. 

—  As  with  any  simulation,  differences  were  noted,  but  test  participants  still  characterized 
VSTARS  as  having  the  same  capabilities  and  limitations  as  Joint  STARS.  They  could 
not  identify  any  VSTARS  process  or  function  that  limited  their  ability  to  perform  their 
mission/job  or  altered  their  approach  to  their  mission/job.  Results  demonstrated  a 
close  correspondence  between  VSTARS  and  Joint  STARS. 

Conclusion  for  JADS  Measure  l-l-O-l.  The  Phase  2  ADS  configuration  produced  valid  SUT 
data.  The  credibility  of  the  SUT  data  was  enhanced  by  the  use  of  actual  operational  hardware, 
wherever  possible  (e.g.,  OWS,  LGSM,  AFATDS),  with  actual  trained  operators.  VSTARS  was 
designed  so  that  the  only  simulated  portion  is  the  simulation  of  the  MTI  and  SAR  radar  modes 
within  the  radar  subsystem.  Everything  else  is  either  integration  code  or  actual  E-8C  system 
code.  The  inputs  into  VSTARS,  except  for  the  target  data,  are  normal  inputs  into  the  real  radar 
processor,  and  the  outputs  are  the  actual  radar  reports.  The  radar  simulations  are  parallel 
processes  with  the  radar  when  live  and  virtual  are  mixed  and  solve  the  radar  equations  in  order  to 
achieve  the  required  fidelity. 

Conclusion  for  JADS  Measure  1-1-0-2.  Under  normal  operations,  aU  input  data  provided  to  the 
SUT  (VSTARS  and  LGSM)  by  the  ADS  environment  were  valid.  Network  performance  and 
reliability  in  delivering  data  to  the  SUT  are  analyzed  under  JADS  Objective  2-1. 

Since  the  Phase  2  ADS  configuration  produced  valid  SUT  data,  the  utility  of  this  configuration 
for  Joint  STARS  DT&E  and  OT&E  was  evaluated. 

Utility  for  DT&E 
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This  architecture  allows  the  use  of  VSTARS  to  conduct  developmental  testing  of  all  of  the  other 
subsystems  that  comprise  Joint  STARS,  provided  that  VSTARS  is  an  accurate  representation  of 
the  radar.  Obviously,  VSTARS  can  not  be  used  to  conduct  developmental  testing  of  the  radar 
subsystem. 

As  an  example,  one  of  the  features  of  the  workstation  used  on  the  E-8C  is  an  automatic  tracker 
(A-tracker).  The  A-tracker  works  off  radar  reports  and,  when  initiated,  will  automatically  track  a 
designated  formation,  providing  bearing,  speed  and  number  of  vehicles.  Prior  to  VSTARS,  it  was 
necessary  to  either  have  a  functioning  radar  (test  flight)  or  a  recording  of  a  functioning  radar  in 
order  to  test  the  A-tracker.  Test  cases  were  basically  limited  to  those  that  could  be  achieved  at 
EgUn  Air  Force  Base,  Florida,  with  a  minimal  number  of  vehicles  traveling  under  peacetime  safety 
restrictions. 

Northrop  Grumman  is  currently  developing  an  annual  release  of  the  radar  software  that  will 
incorporate  a  revised  version  of  the  A-tracker  software.  This  provided  IADS  with  the 
opportunity  to  ask  Northrop  Grumman  to  conduct  an  ad  hoc  study,  parallel  to  the  normal  testing 
of  the  new  A-tracker  software,  to  determine  if  it  would  be  possible  to  use  VSTARS  to  test  this 
software. 

Numerous  problems  in  the  areas  of  software  integration  and  scenario  generation  were  experienced 
because  of  the  ad  hoc  nature  of  the  study.  Additionally,  the  V&V  of  VSTARS  had  not  been 
completed  and  therefore  no  results  could  be  used  as  documentation  for  the  annual  release. 
Despite  these  problems,  several  lessons  were  learned  from  this  study  relating  to  utility  of 
VSTARS  for  DT&E. 

-  Test  cases  could  be  “flown”  using  VSTARS  whenever  needed  with  as  many  repetitions  as 
desired.  This  was  possible  without  competing  for  scarce  test  aircraft  and  range  resources. 

-  There  was  a  potential  for  enormous  cost  savings.  One  tester  and  two  computers  in  a  lab  vice 
the  aircraft,  testers,  crew  and  range  assets  required  for  a  live  test. 

-  Any  conceivable  test  case  could  be  “flown”  in  the  laboratory  without  worr)dng  about  safety  or 
limited  assets  provided  the  appropriate  scenario  generator  was  available. 

-  Bad  software  could  be  quickly  discarded  and  new  software  could  be  tried  the  next  day. 

-  Most  importantly,  when  a  live  test  is  flown,  as  it  must  be,  the  testers  can  be  reasonably  sure 
that  they  will  get  the  maximum  value  from  the  flight  and  test  conditions. 
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utility  for  OT&E 


The  utility  of  this  configuration  for  Joint  STARS  OT&E  was  evaluated  by  determining  which 
measures  from  the  Joint  STARS  Multiservice  Operational  Test  and  Evaluation  (MOT&E)  Plan* 
could  be  supported.  Appendix  B  (available  under  separate  cover  from  JADS  JTF)  identifies 
which  Joint  STARS  MOT&E  measures  could  be  evaluated  using  the  Phase  2  ADS  configuration. 
For  comparison,  the  measures  which  were  addressed  during  the  contingency  operations^  were 
also  identified. 

Results  in  Appendix  B  are  summarized  as  follows. 

-  The  measures  for  critical  operational  issue  (COI)-l  (Does  Joint  STARS  perform  its  tactical 
battlefield  surveillance  mission?)  involving  the  performance  of  the  E-8C  radar  in  its 
operational  environment  cannot  be  evaluated.  In  Phase  2,  a  simulator  is  used  to  represent  the 
E-8C  radar  subsystem.  As  a  result,  the  13  radar  measures  of  performance  (MOPs)  could  not 
be  evaluated  in  the  Phase  2  configuration.  The  Phase  2  configuration  can  be  used  to  evaluate 
4  out  of  5  nonradar  MOPs  supporting  COI-1.  However,  aU  three  measures  of  effectiveness 
(MOEs)  for  COI-1  can  at  least  be  partially  evaluated  using  the  Phase  2  configuration. 

-  As  for  COI-1,  the  measures  for  COI-2  (Does  Joint  STARS  support  the  execution  of  attacks 
against  detected  targets?)  involving  the  performance  of  the  E-8C  radar  in  its  operational 
environment  cannot  be  evaluated  using  the  Phase  2  configuration.  The  Phase  2  configuration 
can  be  used  to  evaluate  8  out  of  13  nonradar  MOPs  supporting  COI-2.  However,  aU  three 
MOEs  for  COI-2  can  at  least  be  partially  evaluated  using  the  Phase  2  configuration. 

-  The  Phase  2  ADS  configuration  can  support  evaluation  of  the  single  MOE  for  COI-3  (Does 
Joint  STARS  provide  timely  and  accurate  information  to  support  battlefield  management  and 
target  selection?)  and  1  out  of  2  MOPs  for  COI-4.  (Can  the  Joint  STARS  system  be  sustained 
in  an  operational  environment?) 

-  The  Phase  2  ADS  configuration  can  support  the  evaluation  of  6  out  of  13  of  the  additional 
nonradar  effectiveness  measures. 

-  Since  the  Phase  2  ADS  configuration  utilizes  an  actual  ground  station  module  (GSM),  8  out 
of  27  suitability  MOPs  involving  the  GSM  could  be  evaluated. 

-  The  Phase  2  ADS  configuration  does  not  involve  the  actual  air  platform  and  could  address 
only  1  out  of  8  human  factors  measures. 


‘  Joint  Surveillance  Target  Attack  Radar  System  (Joint  STARS)  Multiservice  Operational  Test  and  Evaluation 
(MOT&E)  Plan,  Air  Force  Operational  Test  and  Evaluation  Center,  Kirtland  Air  Force  Base,  New  Mexico,  21 
February  1995. 

^  Joint  Surveillance  Target  Attack  Radar  System  (Joint  STARS)  Contingency  Operations  Test  and  Evaluation 
(COT&E)  Plan,  Air  Force  Operational  Test  and  Evaluation  Center,  Kirtland  Air  Force  Base,  New  Mexico,  1 
December  1995. 
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-  The  Phase  2  ADS  configuration  cannot  address  any  of  the  six  software  measures  since  an 
operational  Joint  STARS  software  load  was  not  used. 

In  summary,  the  Phase  2  ADS  configuration  could  evaluate  15  out  of  45  effectiveness  MOPs 
(including  two  MOPs  not  evaluated  during  the  contingency  operations)  and  aU  eight  effectiveness 
MOEs.  Further,  the  Phase  2  ADS  configuration  could  be  used  to  evaluate  the  GSM  suitability 
MOPs  (8  out  of  27  suitability  MOPs).  However,  this  configuration  would  not  be  useful  for 
evaluating  the  human  factors  or  software  measures. 

4.1.2  JADS  Objective  1-2.  Assess  the  benefits  of  using  ADS,  including  DIS,  in  T&E. 

4. 1.2.1  JADS  Subobjective  1-2-2.  Assess  ADS  capability  to  support  T&E  planning  and  test 
rehearsal. 

JADS  Measure  1-2-2-4.  Degree  to  which  pretest  exercise  of  data  reduction  and  analysis 
routines  using  ADS  improved  test  preparations. 

This  measure  evaluated  the  impact  of  ADS  technology  on  data  reduction  and  analysis  routines.  In 
support  of  this  test  question,  the  ETE  Test  analysts  conducted  interviews  with  test  team  members 
involved  in  data  reduction  and  analysis  during  the  Phase  2  test  and  test  rehearsals. 

The  Phase  2  data  reduction  and  analysis  methodology  was  based  on  procedures  successfully  used 
m  earlier  JADS  tests.  Procedures  used  during  the  FI  and  risk  reduction  tests,  as  weU  as  during  the 
Phase  2  operational  test  (OT),  were  basically  unchanged  from  those  used  previously. 

Table  2  describes  the  estimated  number  of  hours  spent  on  rehearsing  data  reduction  and  analysis 
procedures  in  each  of  the  functionality  and  integration  (FI)  tests,  as  well  as  during  the  risk 
reduction  test.  These  rehearsals  were  focused  on  ensuring  that  the  proper  tools  and  procedures 
were  in  place  for  each  pretest.  Phase  2  pretest  activities  provided  several  occasions  for  JADS 
analysts  to  practice  the  data  reduction  and  analysis  procedures  used  later  in  the  operational  test. 
The  times  presented  in  Table  2  do  not  include  time  spent  on  either  data  collection  or  report 
writing. 

Data  reduction  and  analysis  activities  in  Phase  2  were  aided  by  the  use  of  the  JADS  Analysis 
Toolbox.  Developed  over  the  course  of  earlier  JADS  tests,  the  toolbox  is  a  set  of  software 
routines  written  in  the  C++  programming  language  and  integrated  into  a  single  user  interface. 

The  toolbox  can  be  employed  to  develop  tables  and  plots  of  PDU  data  in  near  real  time.  After  the 
test,  toolbox  users  can  replay  or  get  selected  data  in  a  text-readable  format  from  JADS  logfiles 
and  obtain  plots  and  tables  of  PDU  statistics.  The  PDU  statistics,  in  turn,  include  the  number  of 
PDUs  received,  the  number  of  each  type  of  PDU  received,  and  the  rate  at  which  PDUs  are  being 
received.  An  important  point  is  that  the  ETE  Test  logfiles  were  larger  than  earlier  JADS  test 
logfiles.  Modifications  were  made  to  the  toolbox  software  to  minimize  the  impact  of  the  size  of 
the  ETE  Test  logfiles  on  file  access  time,  as  well  as  on  the  amount  of  memory  needed  for 
calculations.  (For  further  information  on  the  JADS  Analysis  Toolbox,  please  contact  the  JADS 
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program  office  or  visit  the  JADS  website  at  http://www.jads.abq.com  where  it  will  be  available 
until  September  2000.  After  that  it  will  be  available  at  HQ  AFOTEC/HO,  8500  Gibson  Blvd  SE, 
Kirtland  Air  Force  Base,  New  Mexico  87117-5558  and  at  the  SAIC  Technical  Library,  2001 
North  Beauregard  St.  Suite  800,  Alexandria,  Virginia  22311.) 

Table  2.  Time  Spent  on  Pretest  Data  Reduction  and  Analysis  Rehearsals 


Data  Type 

FI#1 

FI  #2 

Risk  Reduction 
Test 

PDU  Data 

8  hrs 

8  hrs 

8  hrs 

8  hrs 

16  hrs 

Latency 

8  hrs 

8  hrs 

8  hrs 

8  hrs 

16  hrs 

Network 

Availability 

4  hrs 

4  hrs 

4  hrs 

ADS  System 
Availability 

4  hrs 

4  hrs 

4  hrs 

Bandwidth 

2  hrs 

1^^311111 

3  hrs 

Packet  Rate 

1  hr 

1  hr 

3  hrs 

Other  OT 
Test 

Measures 

60  hrs 

The  ETE  Test  Phase  2  also  demonstrated  the  unique  advantages  of  an  ADS  environment  with 

respect  to  data  reduction  and  analysis  processes. 

-  For  both  ADS  and  conventional  tests,  these  routines  are  typically  developed  prior  to  the 
actual  test  events.  For  non- ADS  tests,  actual  test  assets  (i.e.,  flight  time)  may  need  to  be 
expended  in  order  to  produce  the  data  needed  to  test  these  routines.  In  contrast,  the  JADS 
analysts  were  able  to  exercise  their  data  reduction  and  analysis  routines  using  data  from  the  FI 
and  risk  reduction  tests  accomplished  prior  to  Phase  2  OT  testing.  This  early  availability  of 
test  data  enabled  the  analysts  to  check  out  and  refine  these  procedures  and  prevented  the  loss 
of  critical  test  data  from  the  actual  Phase  2  OT  test. 

-  If  an  ADS  test  team  emphasizes  the  early  exercise  of  data  reduction  and  analysis  routines 
during  pretest  rehearsals,  it  can  reduce  or  eliminate  the  loss  of  critical  data  and  expenditure  of 
valuable  test  assets  during  subsequent  actual  testing. 

-  In  addition,  the  pretest  rehearsals  offered  testers  the  opportunity  to  improve  the  data 
gathering  process.  For  example,  when  beginning  Phase  2,  the  ETE  Test  team  intended  to  log 
all  operator  actions  in  the  LGSM.  As  the  test  progressed,  the  JADS  analysts  realized  that 
their  efforts  to  obtain  data  from  the  LGSM  operators  were  hampering  the  latter’s  abilities  to 
provide  accurate  and  timely  information  to  the  TAG.  As  a  result,  the  JADS  analysts 
automated  most  of  the  data  collection  and  retrieved  it  at  the  end  of  the  mission. 


36 


-  The  networked  nature  of  an  ADS  environment  provides  a  built-in  means  to  support  data 
management.  During  the  FI  and  risk  reduction  tests  prior  to  the  Phase  2  operational  test,  the 
ETE  Test  team  took  advantage  of  this  capability  and  was  able  to  speed  the  transport  of  test 
data  from  the  nodes  to  a  central  collection  and  analysis  point. 

JADS  Measures  1-2-2-5  and  1-2-3-24.  Degree  to  which  ADS  can  he  used  for  tactics 
development  prior  to  test  execution.  Degree  to  which  ADS  can  ensure  that  correct 
techniques  and  procedures  are  used. 

These  measures  determined  if  the  techniques  and  procedures  used  by  the  test  participants  to 
obtain  and  disseminate  information  on  target  enemy  forces  could  be  enhanced  in  any  way  using  an 
ADS  environment.  In  a  broader  context,  JADS  analysts  wanted  to  determine  if  the  ADS  test 
environment  could  be  used  to  correct,  refine,  and  change  tactics,  techniques,  and/or  procedures 
prior  to  exercising  a  hve  test.  During  the  rehearsal  tests,  prior  to  the  Phase  2  operational  test, 
JADS  analysts  recorded  the  flow  of  information  from  the  TAC  to  the  LGSM  and  aU  fire  missions 
initiated  by  the  TAC  battle  captain  and  executed  using  ATACMS.  After  each  of  these  tests,  a 
follow-up  interview  was  conducted  with  the  TAC  battle  captain  to  determine  the  targeting 
process  used  and  to  assess  the  potential  of  using  ADS  to  improve  current  techniques  and 
processes  and  develop  tactics. 

ADS  testing  also  allowed  the  operators  to  use  the  tactical  scenarios  and  make  real-world 
decisions  concerning  their  surveillance  and  targeting  efforts.  The  JADS  analysts  noticed  a  great 
improvement  over  time  in  the  operators’  abilities  to  focus  on  certain  aspects  of  the  scenario  and  in 
their  increased  use  of  the  tools  available  to  them  to  obtain  more  information  on  the  entities  they 
were  observing. 

The  increased  test  time  provided  by  an  ADS  environment  appears  valuable  in  allowing  C4ISR 
system  operators  and  end  users  to  confirm  and  refine  current  tactics  and  to  experiment  with  “what 
if’  scenarios  and  new  tactics.  This  feature  will  be  further  examined  in  the  Phase  4  test. 

4. 1.2.2  JADS  Subobjective  1-2-3.  Assess  ADS  capability  to  support  T&E  shortfalls. 

JADS  Measures  1-2-3-1, 1-2-3-17,  and  1-2-3-28.  Degree  to  which  ADS  can  add  test  articles 
to  test  execution,  provide  for  more  representative  force  levels,  and  increase  the  number  of 
simulation  entities. 

Conventional  T&E  typically  suffers  from  an  inadequate  number  of  entities.  Typical  tests  may 
have  to  be  conducted  in  conjunction  with  training  missions  in  order  to  acquire  possibly  hundreds 
of  assets.  However,  testers  will  then  have  only  minimal  control  over  test  specifics  and  must  base 
their  test  on  the  training  scenario  being  executed. 

In  contrast.  Table  3  shows  that  the  Janus  simulation  provided  thousands  of  entities  for  each 
vignette  in  Phase  2. 
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Phase  2  results  imply  the  following  benefits  of  ADS-based  testing. 

-  An  ADS  environment  using  vahdated  simulations  can  provide  more  realistic  force  levels  than 
those  offered  by  conventional  tests,  i.e.,  levels  otherwise  unavailable  because  of  cost 
restrictions. 

-  C4ISR  system  testers  can  tailor  the  simulation  entities  operating  in  the  ADS  environment  to 
closely  reflect  the  forces  expected  in  operational  theaters,  thus  further  increasing  the  relevance 
of  the  collected  test  data. 

-  ADS  allows  testers  to  have  more  control  over  the  specific  aspects  of  the  scenarios  of  interest 
and  to  expand  their  test  concept  and  design.  A  typical  constraint  to  test  concept  development 
is  the  number  and  types  of  units  readily  available  for  a  test.  For  example,  a  conventional  test 
may  require  the  use  of  a  battalion  of  troops,  but  because  of  a  prior  commitment  or  cost,  these 
personnel  may  not  be  available.  In  contrast,  ADS  allows  testers  to  create  a  virtual  battalion 
and  to  test  their  concepts  with  a  minimum  number  of  personnel  and  equipment. 

Table  3.  Entity  Count  Per  Vignette 


Vignette 

Number  of 
Entities 

1 

9,897 

2 

9,757 

3 

9,904 

4 

9,781 

5 

9,950 

JADS  Measure  1-2-3-3.  Degree  to  which  ADS  overcomes  performance  restrictions. 

ADS  is  helpful  in  simulating  maneuver  scenarios  that  overcome  shortfalls  associated  with 
traditional  testing.  This  technology  increases  test  robustness  by  providing  the  capability  for 
adding  large  numbers  of  assets.  ADS  simulations  can  depict  such  unsafe  conditions  as  convoy 
vehicles  driving  across  rugged  or  restricted  terrain  under  wartime  conditions.  Finally,  this 
technology  frees  the  tester  from  the  constraints  of  environmental  restrictions.  Without  the 
benefits  of  ADS-enhanced  testing,  it  would  be  almost  impossible  to  instrument  and  maneuver  a 
corps-size  element. 

JADS  Measure  1-2-3-4.  Degree  to  which  ADS  overcomes  the  traditional  T&E  shortfall  of 
insufficient  battlespace. 

ADS  technology  can  easily  eliminate  the  conventional  testing  disadvantage  of  insufficient 
battlespace.  For  example,  the  National  Training  Center  occupies  about  one  thousand  square 
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miles.  In  contrast,  the  battlespace  for  Phase  2  of  the  EXE  Test  was  almost  ten  times  larger.  In 
fact,  ADS  technology  is  capable  of  supporting  even  larger  battlespaces. 

JADS  Measures  1-2-3-7  and  1-2-3-12.  Degree  to  which  ADS  overcomes  the  lack  of  systems 
for  interoperahility  testing  associated  with  traditional  T&E  and  can  increase  the  number  of 
systems  for  compatibility  evaluation. 

There  were  many  incompatibility  problems  among  the  fielded  real  systems  that  persisted  well  into 
the  risk  reduction  test.  The  TAG  had  never  had  the  opportunity  to  perform  this  function,  under 
tactical  conditions  in  a  doctrinally  correct  manner,  prior  to  the  risk  reduction  test.  Use  of  the  risk 
reduction  test  ADS  environment  allowed  these  incompatibility  problems  to  be  identified  and 
resolved. 

Phase  2  results  show  that  ADS  can  link  similar  systems  to  perform  simultaneous  testing  and 
training  at  various  locations.  The  ETE  Test  Phase  2  used  only  one  LGSM  to  provide  intelligence 
to  the  TAG  at  Fort  Hood.  However,  the  potential  exists  for  linking  numerous  LGSMs  at  different 
military  installations  and  simultaneously  conducting  the  same  type  of  test  and  training. 

JADS  Measure  1-2-3-9.  Degree  to  which  ADS  increases  the  number  of  test  events. 

Using  ADS,  testers  can  replicate  a  test,  as  well  as  test  for  longer  periods  of  time.  During  the 
Phase  2  test,  the  ETE  Test  team  was  able  to  test  for  thirty  hours  within  a  5-day  period.  A  test 
using  real  assets,  and  of  similar  duration,  would  be  very  expensive  and  probably  even  impossible. 

JADS  Measure  1-2-3-31.  Degree  to  which  ADS  can  represent  joint/combined  operations 
and  capabilities. 

ADS  can  reahsticaUy  portray  joint  operations  among  nuhtary  elements  of  the  same  nation.  For 
example,  the  Phase  2  test  employed  a  mix  of  Air  Force  and  Army  personnel  located  at  different 
facUities  successfully  simulating  a  G4ISR  system  interacting  with  ground-based  units.  Given  the 
necessary  planning  and  resources,  ADS  could  also  represent  combined  operations  between  forces 
of  two  or  more  allies. 

JADS  Measure  1-2-3-36.  Degree  to  which  ADS  can  increase  personnel  resources. 

The  duties  of  all  personnel  involved  in  the  setup  and  execution  of  the  Phase  2  test  were  examined 
to  determine  which  personnel  would  not  have  been  required  for  a  traditional  test.  In  addition,  the 
number  of  personnel  participating  in  the  test,  due  solely  to  ADS-specific  requirements,  was 
documented. 

The  ETE  Test  Phase  2  results  did  not  provide  any  set  formula  for  forecasting  personnel  needs. 
Rather,  the  exact  personnel  requirements  for  G4ISR  system  testing  in  an  ADS  environment 
appears  to  vary  from  test  to  test.  With  this  caveat  in  mind,  and  using  just  the  ETE  Test  team 
requirements,  an  ADS  environment  does  not  appear  to  add  to  the  persormel  needs  of  a  G4ISR 
test.  Six  JADS  personnel  are  continually  involved  in  ETE  Test  management  and  planning,  a 
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number  equivalent  to  the  staff  needed  to  direct  the  day-to-day  operations  of  a  conventional 
C4ISR  test  of  the  same  magnitude.  Thirteen  people  were  involved  in  test  monitoring  and  data 
collection  at  the  various  ETE  Test  nodes  during  the  actual  execution  of  the  Phase  2  FI,  risk 
reduction,  and  operational  tests.  Given  the  distributed  nature  of  C4ISR  systems,  a  similar  number 
of  people  would  be  needed  in  a  comparable,  traditional  C4ISR  test.  Two  Network  and 
Engineering  persoimel  were  needed  for  network  support  during  the  execution  of  Phase  2,  and  two 
analysts  handled  the  subsequent  data  analysis  and  reduction.  However,  a  conventional  C4ISR 
SUT  would  probably  need  at  least  two  network  engineers  because  of  its  distributed  nature,  and 
maybe  more,  since  it  might  not  be  as  reliable  as  the  ETE  Test  Phase  2  network.  In  addition,  the 
C4ISR  environment  would  likewise  require  a  minimum  of  two  data  analysts. 

4.2  JADS  Issue  2.  What  are  the  critical  constraints,  concerns,  and  methodologies 
when  using  ADS  for  T&E? 

The  ETE  Test  Phase  2  demonstrates  that  there  are  no  real  technical  barriers  to  using  an  ADS 
environment  to  provide  a  realistic  test  environment  for  a  C4ISR  system.  This  is  due  to  the  high 
reliability  of  the  network  architecture  underlying  an  ADS  environment  and  the  dramatic  increases 
in  computer  processing  and  storage  capabilities  over  the  past  few  years.  Rather,  the  key 
constraints  to  ADS  testing  are  time  and  money:  How  soon  do  you  need  results?  How  much  are 
you  willing  to  spend  on  development? 

The  following  paragraphs  discuss  constraints  and  concerns  in  detail. 

4.2.1  JADS  Objective  2-1.  Assess  the  critical  constraints  and  concerns  in  ADS 
performance  for  T&E. 

4.2. 1.1  JADS  Subobjective  2-1-2.  Assess  network  and  communications  performance  constraints 
and  limitations. 

JADS  Measures  2-1-2-2  and  2-1-3-3.  Percentage  of  ADS  trials  canceled  or  otherwise  not 
used  due  to  network  problems.  Percentage  of  trials  in  which  network  connections  were  lost 
long  enough  to  require  trial  cancellation. 

For  these  measures,  the  network  was  defined  as  including  all  software  and  hardware  used  for 
connecting  the  distributed  sites  and  aU  loggers  and  instrumentation  used  for  recording  network 
data.  NIUs  were  considered  part  of  the  individual  simulations  and  not  part  of  the  network. 

For  each  trial,  an  execution  log  was  maintained  at  each  node.  The  data  collectors  annotated  aU 
problems  encountered,  as  well  as  their  causes.  A  test  controller  log  also  documented  the  overall 
status  of  the  network  and  test  trials.  In  addition,  network  monitoring  tools  were  used  to  monitor 
the  status  of  aU  network  links  between  nodes.  Any  problems  detected  by  the  monitoring  tools 
were  documented  via  line  printers  in  terms  of  a  brief  explanation  of  the  problem,  the  time,  and  the 
link(s)  involved. 
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No  trials  were  canceled,  or  not  used,  because  of  network  problems.  However,  two  trials  were 
postponed  when  the  agency  contracted  to  provide  network  service  inadvertently  terminated  use  of 
the  T-1  line  at  the  Northrop  Grumman  node.  The  coincidence  of  a  hurricane  hitting  the  Florida 
and  Gulf  coasts  during  the  same  timeframe  delayed  the  reactivation  of  this  link.  As  a  result,  the 
test  was  extended  two  days  after  the  scheduled  test  period.  Table  4  shows  the  dates  of  the  trials 
and  their  test  times. 


Table  4.  ETE  Test  Phase  2 


Trial 

Vignette 

Test  Time 

Comments 

28  Sep 

Postponed  to  5  Oct 

29  Sep 

Postponed  to  6  Oct 

30  Sep  (Day  1) 

3 

7  hrs,  1  min 

1  Oct  (Day  2) 

4 

7  hrs,  4  mins 

2  Oct  (Day  3) 

5 

7  hrs,  7  mins 

5  Oct  (Day  4) 

1 

7  hrs 

6  Oct  (Day  5) 

2 

7  hrs,  15  mins 

JADS  Measure  2-1-2-3.  Average  and  peak  bandwidth  used  by  link. 

This  measure  provided  an  indication  of  bandwidth  use  and  packet  rate  during  the  OT.  Although 
bandwidth  utilization  was  not  expected  to  exceed  capacity,  the  utilization  rate  was  documented  to 
provide  other  ADS  testers  with  an  indication  of  the  amount  of  needed  bandwidth.  The  packet 
rate  data  are  also  included  because  of  their  potential  value  to  other  ADS  testers. 

Data  were  collected  using  the  Spectrum™  network  analysis  tool.  Spectrum™  provided  the 
capability  to  study  multiple  aspects  of  network  link  performance,  including  packet  rate  and 
percentage  of  bandwidth  utilized.  A  polling  rate  of  fifteen  seconds  was  used  in  the  collection  of 
these  data. 

Once  aU  the  data  were  gathered,  the  JADS  analysts  consolidated  the  data  by  network  Unk.  These 
data  were  then  used  to  calculate  daily  packet  rate  and  bandwidth  values  (maximum  and  average) 
for  each  link.  The  bandwidth  values  were  provided  by  Spectrum™  as  the  percentage  of 
bandwidth  available  on  the  T-1  Une.  A  T-1  line  has  a  normal  bandwidth  of  1.544  megabits  per 
second  (Mbps).  For  the  ETE  Test,  some  of  the  bandwidth  of  the  T-1  line  was  reserved  for  voice 
traffic,  leaving  a  maximum  bandwidth  available  of  1.344  Mbps. 

Table  5  shows  average  and  maximum  performance  values  for  the  classified  network  links 
monitored  during  the  five  days  of  active  ETE  Phase  2  testing. 

Packet  rate  and  bandwidth  utilized  remained  fairly  constant  across  the  five-day  test  period.  The 
packet  rate  and  bandwidth  utilization  rate  experienced  over  the  TCAC-Northrop  Grumman  link 
averaged  approximately  18  packets  per  second  and  less  than  1%,  respectively.  The  packet  rate 
experienced  over  the  Northrop  Grumman-Fort  Hood  link  averaged  approximately  34  packets  per 
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second  with  an  average  utilization  rate  of  about  1%  of  capacity.  The  maximum  packet  rate  over 
the  TCAC-Northrop  Grumman  link  was  120  packets  per  second  resulting  in  a  20%  peak  load 
experienced.  However,  these  maximum  values  appeared  to  be  anomalous,  caused  by  the 
reestablishment  of  the  Janus  heartbeat  following  a  crash;  the  more  typical  maximum  values  were 
37  packets  per  second  and  2%  bandwidth  utilization.  The  maximum  packet  rate  experienced 
over  the  Northrop  Grumman-Fort  Hood  link  was  102  packets  per  second  resulting  in  a  5% 
bandwidth  utilization  rate.  These  maximum  values  were  consistent  from  day  to  day.  These  data 
for  the  two  links  show  the  relatively  small  bandwidth  utilization  experienced  during  the  Phase  2 
test  with  the  maximum  packet  rate  experienced  still  leaving  nominally  95%  of  the  1.344  Mbps  of 
bandwidth  available  for  use.  Future  ETE  tests  will  use  the  available  bandwidth  to  facilitate  a 
higher  heartbeat  by  sending  more  frequent  entity  state  PDU  updates  from  the  Janus  simulation. 

Table  5.  Link  Performance  Characteristics* 


Day 

Node  A 

Node  B 

Load 

Average  Maximum 

Packet  Rate 

Average  Maximum 

1 

T 

G 

0.33% 

16.93/sec 

34.0/sec 

G 

H 

0.66% 

29.18/sec 

98.0/sec 

2 

T 

G 

0.87% 

18.60/sec 

34.0/sec 

G 

H 

0.98% 

5.0% 

34.64/sec 

102.0/sec 

3 

T 

G 

0.63% 

20.0% 

18.07/sec 

120.0/sec 

G 

H 

1.27% 

5.0% 

36.64/sec 

97.0/sec 

4 

T 

G 

0.54% 

2.0% 

17.99/sec 

37.0/sec 

G 

1.43% 

5.0% 

39.39/sec 

97.0/sec 

5 

T 

0.53% 

2.0% 

18.0/sec 

33.0/sec 

G 

H 

0.99% 

5.0% 

28.87/sec 

98.0/sec 

Total 

T 

G 

0.58% 

20.0% 

17.92/sec 

120.0/sec 

G 

H 

1.07% 

5.0% 

33.74/sec 

102.0/sec 

T  =  TCAC  G  =  Northrop  Grumman  H  =  Fort  Hood 

*  Table  refers  only  to  active  test  time  during  which  PDU  loggers  were  recording  data. 


JADS  Measures  2-1-2-5,  2-1-2-6,  and  2-1-2-7.  Percentage  of  time  PDUs  were  received  out 
of  order  by  a  network  node.  Percentage  of  total  PDUs  required  at  a  node  that  were 
delivered  to  that  node.  Average  and  peak  data  latency  between  ADS  nodes. 

The  flow  of  PDUs  to  and  from  each  node  was  recorded  using  loggers  installed  as  part  of  the 
network  architecture.  The  loggers  specifically  recorded  the  time  and  order  that  the  PDUs  were 
transmitted  and  received  at  each  node. 
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The  raw  logger  data  were  transformed  and  reduced  for  analysis  to  determine  out  of  order  PDUs, 
duplicate  PDUs,  lost  PDUs,  and  PDU  latency.  These  data  were  then  used  to  calculate  the 
percentage  of  out  of  order,  duphcate,  and  lost  PDUs  at  each  node  for  each  vignette  and  for  the 
test  as  a  whole.  The  minimum,  maximum,  and  mean  latency  of  PDUs  were  also  computed.  JADS 
analysts  accomphshed  these  calculations  using  UNIX®-based  software  tools  created  by  JADS 
programmers.  Results  for  these  measures  are  given  in  Tables  6  and  7. 

Table  6.  Vignette  PDU  Data 


43 


W=  WSMR  T  =  TCAC  G  =  Northrop  Grumman 


=  Fort  Sill 


44 


Table  7.  Vignette  Latency  Data 


Vignette 

Node  A 

NodeB 

Latency  (seconds)  | 

Minimum 

Mean 

Maximum 

1 

w 

T 

0.020 

0.040 

0.145 

T 

G 

0.051 

0.056 

0.605 

S 

W 

0.032 

0.034 

0.383 

2 

w 

T 

0.018 

0.037 

0.139 

T 

G 

0.050 

0.057 

1.090 

S 

W 

0.033 

0.034 

0.381 

3 

w 

T 

0.020 

0.041 

0.079 

T 

G 

0.051 

0.058 

1.080 

S 

W 

0.030 

0.035 

0.392 

4 

w 

T 

0.016 

0.039 

0.084 

T 

G 

0.051 

0.056 

0.580 

S 

W 

0.031 

0.036 

0.396 

5 

w 

T 

0.021 

0.038 

0.079 

T 

G 

0.050 

0.059 

0.585 

S 

W 

0.031 

0.034 

0.361 

Total 

w 

T 

0.016 

0.039 

0.145 

T 

G 

0.050 

0.057  I 

1.090 

S 

W 

0.030 

0.035 

0.396 

W=  WSMR  T  =  TCAC  G  =  Northrop  Grumman  S  =  Fort  Sill 


Table  6  shows  the  PDU  data  for  each  vignette  by  node;  there  were  no  duphcate  or  out  of  order 
PDUs.  Table  7  provides  the  latency  data  for  the  vignettes  broken  down  into  the  individual 
network  links  between  nodes.  These  tables  indicate  the  high  rehabUity  of  the  network  in  passing 
PDUs  and  the  network  ability  to  maintain  stable  latencies  during  the  Phase  2  test. 

The  PDU  data  in  Table  6  show  total  PDU  loss  rates  of  0.025,  0.052,  and  0.007  percent  for  the 
WSMR-TCAC,  TCAC-Northrop  Grumman,  and  Fort  Sih-WSMR  links,  respectively.  Note  that 
the  total  loss  rate  for  ESPDUs  generated  by  Janus  being  delivered  from  the  WSMR  node  to  the 
Northrop  Grumman  node  (the  node  requiring  them)  is  0.077  percent  (or  295  PDUs  lost  out  of 
382,254  PDUs  sent).  These  loss  rates  were  insignificant  and  did  not  affect  the  vahdity  of  the 
Phase  2  OT  results. 

The  latency  data  in  Table  7  show  that  the  average  latency  for  a  given  link  was  very  stable  over  the 
five  days  of  testing,  varying  by  only  five  percent  or  less.  Note  that  the  latency  over  the  TCAC- 
Northrop  Grumman  link  had  maximum  values  exceeding  one  second.  However,  these  were  only 
transient  values  and  did  not  significantly  affect  the  average  latency  (which  varied  by  only  about 
3%  over  the  five  days).  The  average  total  latency  for  the  ESPDU  data  to  travel  firom  the  WSMR 
node  to  the  Northrop  Grumman  node  was  less  than  100  nuUiseconds  and  had  no  effect  on  the 
validity  of  the  Phase  2  OT  results. 
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JADS  Measure  2-1-3-6.  Average  downtime  due  to  ADS  network  failures. 

This  measure  identified  the  impact  of  network  failures  on  the  OT.  During  Phase  2,  logs  were  kept 
to  record  all  network  problems,  the  start  time  and  duration  of  the  problems,  and  problem 
resolution.  In  addition,  network  monitoring  tools  were  used  to  monitor  the  status  of  aU  network 
links  between  the  nodes.  Any  problem  detected  by  the  monitoring  tools  was  documented  via  line 
printers  in  terms  of  a  brief  explanation  of  the  problem,  the  time,  and  the  link(s)  involved. 

During  the  five  days  of  operational  testing,  only  three  network  outages  were  experienced, 
resulting  in  8  minutes  of  downtime  (Table  8). 

Table  8.  Network  Downtime 


Day 

Time 

Scheduled  for 
Testing 

Time 

Network 
Unavailable 
for  Testing 

%  of  Time 

Network 

Unavailable 

Reason  Unavailable 

1 

7  hrs,  1  min 

3  mins 

0.71% 

Router  down  at  Fort  Hood 

2 

7  hrs,  4  mins 

0  mins 

0% 

3 

7  hrs,  7  mins 

3  min 

0.70% 

Router  down  at  Northrop 
Grumman 

4 

7  hrs 

0  min 

0% 

5 

7  hrs,  15  mins 

2  mins 

0.46% 

Router  down  at  Fort  Hood 

Total 

35  hrs,  27  mins 

8  mins 

0.38% 

4.2. 1.2  JADS  Subobjective  2-1-3.  Assess  the  impact  of  ADS  reliability,  availability,  and 
maintainability  on  T&E. 

JADS  Measures  2-1-3-1  and  2-1-3-5.  Percentage  of  trials  delayed,  rescheduled,  and/or 
redone  due  to  ADS  systems  (exclusive  of  network)  unavailability.  Mean  operating  time 
between  ADS  system  failures  (severe  enough  to  require  trial  cancellation). 

These  measures  determined  the  availability  of  ADS  nodes  including  the  NIUs  and  the  impact  of 
node  failures  on  Phase  2  testing. 

For  each  trial,  an  execution  log  was  maintained  at  each  node.  The  data  collectors  annotated  aH 
problems  encountered  with  the  ADS  systems,  along  with  their  causes.  A  test  controller  log  was 
also  maintained  to  document  the  overall  status  of  the  trials. 
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Once  the  Phase  2  test  network  was  available,  no  trials  had  to  be  rescheduled.  In  addition,  there 
were  no  significant  delays  and  only  minor  ADS  system  failures.  Table  9  lists  the  reported  ADS 
system  failures,  along  with  the  time  needed  to  resolve  these  interruptions  and  their  impact  on 
testing. 


Table  9.  ADS  System  Failures 


■m 

Failure 

Resolution 

Duration 

Test  Time 

Impact  on  Test 

28  Sep 

Network 

unavailable 

Trial  rescheduled 

29  Sep 

Network 

unavailable 

Trial  rescheduled 

1 

One  LGSM 
console 
crashed  at 

Fort  Hood 

Console 

rebooted 

17  mins 

7  hrs, 

1  min 

Only  one  workstation  in 
use  (minimal  impact);  no 
PDUs  lost 

2 

No  failures 

N/A 

N/A 

7  hrs, 

4  mins 

N/A 

3 

Power  outage 
at 

Fort  Hood 

LGSM 
rebooted 
once  power 

restored 

21  mins 

7  hrs, 

7  mins 

Fire  mission  coordination 
delayed  (minor  impact); 
no  PDUs  lost 

Janus  locked 
upatWSMR 

Janus 

rebooted 

13  mins 

Trial  stopped  and  restarted 
at  point  of  lockup;  no  PDUs 
lost 

VSTARS 
crashed  at 
Northrop 
Grumman 

VSTARS 

rebooted 

10  mins 

Updates  and  coordination 
with  Fort  Hood  interrupted; 
no  PDUs  lost 

4 

Janus  locked 
up  at  WSMR 

Janus 

rebooted 

24  mins 

7  hrs 

Trial  stopped,  then  restarted 
at  point  of  lockup;  no  PDUs 
lost 

Janus  locked 
up  at  WSMR 

Janus 

rebooted 

31  mins 

Trial  stopped,  then  restarted 
at  point  of  lockup;  no  PDUs 
lost 

5 

VSTARS 
crashed  at 
Northrop 
Grumman 

VSTARS 

rebooted 

30  mins 

7  hrs, 

15  mins 

Updates  and  coordination 
with  Fort  Hood  interrupted; 
no  PDUs  lost 

Janus  locked 
up  at  WSMR 

Janus 

rebooted 

23  mins 

Trial  stopped,  then  restarted 
at  point  of  lockup;  no  PDUs 
lost 

Extensive  risk  reduction  testing  was  instrumental  in  preventing  any  major  ADS  system  failures 
fi'om  affecting  the  Phase  2  test.  Numerous  ADS  problems  were  identified  and  resolved  during 
this  risk  reduction  period,  minimizing  the  amount  of  potential  problems  for  the  test.  As  a  result, 
the  ADS  problems  noted  in  Table  9  had  minor  impact  on  Phase  2  testing  and  caused  only  brief 
delays  or  reductions  in  capability. 
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4.2.2  JADS  Objective  2-2.  Assess  the  critical  constraints  and  concerns  in  ADS  support 
systems  for  T&E. 

4.2.2. 1  JADS  Subobjective  2-2-1.  Assess  the  critical  constraints  and  concerns  regarding  ADS 
data  management  and  analysis  systems. 

JADS  Measure  2-2-1-1.  Degree  to  which  ADS  nodes  provide  for  collection,  data  entry,  and 
quality  checking  of  pre-  and  post-trial  briefing  data. 

Quick-look  analysis  of  results  was  used  to  support  the  post-trial  briefings.  This  analysis  relied 
primarily  on  automated  data  collection  at  aU  ETE  Test  nodes.  The  data  collection  tools  included 
the  JADS  logger  which  collected  the  PDU  log  files  and  a  Spectrum™  logger  to  monitor  network 
performance.  Data  collection  tools  were  attached  to  the  network  at  each  node  without  any 
impact  on  network  or  node  performance.  At  the  end  of  each  test  day,  the  data  were  remotely 
retrieved  by  the  TCAC  and  the  file  size  checked.  This  procedure  supported  timely  quick-look 
analysis  and  test  feedback. 

In  addition  to  electronic  data  logs,  manually  written  logs  were  kept  at  each  test  site  and  used  to 
support  post-trial  briefings.  During  the  test  rehearsals  for  Phase  2,  log  sheets  were  faxed  to  the 
TCAC  at  the  end  of  each  test  day.  This  procedure  proved  less  than  effective,  and  a  daily  after¬ 
action  teleconference  caU  was  added.  This  enabled  the  test  controller  to  discuss  and  fully 
understand  the  problems  of  the  day  without  having  to  review  local  log  sheets. 

JADS  Measure  2-2-1-2.  Adequacy  of  relevant  test  data  storage  at  ADS  nodes. 

The  ETE  Test  analysis  requirements  drove  test  data  storage  needs.  The  focus  of  data  analysis  at 
each  site  was  on  network  latency,  as  well  as  the  actual  PDU  input  or  data  output  at  each  site.  The 
need  to  record  PDU  traffic  at  each  node  required  a  determination  of  the  data  output  and  reception 
rates  at  aU  sites.  The  largest  contributor  to  ESPDU  traffic  was  the  output  of  the  Janus  simulation. 
ESPDUs  from  Janus  are  a  function  of  the  Janus  heartbeat  and  the  vignette  design.  During  the 
Phase  2  testing,  the  Janus  heartbeat  was  set  to  update  all  entities  every  eleven  minutes  during  the 
first  hour.  In  addition,  Janus  had  to  output  an  ESPDU  when  an  entity  changed  state,  i.e.,  start, 
stop,  turn,  etc.  As  a  result,  the  ESPDU  output  grew  as  the  number  of  movers  increased.  The 
ETE  Test  used  five  different  vignettes,  ranging  from  prehostility  with  low  numbers  of  movers  to 
an  active  battle  vignette  with  more  than  3,000  entities  moving  at  one  time.  Prior  to  Phase  2 
testing,  the  five  vignettes  were  played  and  the  ESPDU  output  recorded.  The  maximum  file  size 
seen  during  this  testing  was  about  fifteen  megabytes.  To  support  the  data  recording  as  well  as  file 
storage  and  local  software  requirements,  the  JADS  Network  and  Engineering  team  installed  4- 
gigabyte  hard  drives  on  the  SGI  Indy  at  each  node. 

During  preparations  for  the  Phase  2  test,  the  Northrop  Grumman  node  required  the  largest  data 
capacity  in  order  to  support  VST ARS  software  testing  in  a  stand-alone  mode.  This  testing 
required  the  playback  of  PDU  files  recorded  from  TRAC-WSMR  to  VSTARS.  AU  five  vignettes 
were  played  back  at  various  times,  and  at  least  five  vignette  PDU  files  were  stored  on  the  SGI 
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Indy  at  aU  times.  During  actual  Phase  2  testing,  the  ETE  Test  team  found  hard  drive  data  storage 
capacity  to  he  more  than  adequate. 

The  development  of  data  storage  needs  required  a  full  understanding  of  each  node’s  requirements. 
Since  the  cost  of  hard  drive  storage  has  decreased  dramatically  over  the  past  few  years,  it  was 
cost  effective  to  allow  for  unexpected  growth  by  significantly  exceeding  the  expected  storage 
requirements. 

JADS  Measure  2-2-1-4.  Ease  with  which  data  can  be  retrieved  post  trial  from  a  given 
node. 

During  Phase  2  of  the  ETE  Test,  automated  data  were  collected  using  PDU  loggers  at  the  nodes, 
and  operational  data  were  collected  using  log  sheets  at  each  node.  The  automated  data  collected 
during  the  test  were  retrieved  from  the  JADS  test  locations  by  the  TCAC  using  FTP,  while  JADS 
personnel  transported  the  log  sheets  to  JADS.  The  automated  data  were  compressed  and 
converted  for  analysis  using  JADS  UNIX®-based  tools. 

The  automated  data  retrieval  process  was  very  effective.  For  the  ETE  Test  Phase  2,  the 
automated  data  retrieval  process  needed  about  thirty  minutes  despite  the  large  sizes  of  the  log 
files  being  converted.  Also,  there  were  no  problems  encountered  during  any  of  the  retrieval 
efforts.  As  exemplified  by  the  ETE  Test  team’s  data  retrieval  process,  any  ADS  data  retrieval 
methodology  need  not  be  complex. 

4.2.2.2  JADS  Subobjective  2-2-2.  Assess  the  critical  constraints  and  concerns  regarding 
configuration  management  of  ADS  test  assets. 

JADS  Measure  2-2-2-1.  Degree  to  which  test  managers  can  control  the  configurations  of 
ADS  participants,  the  ADS  environment  data,  and  ADS  networks. 

This  measure  determined  if  the  test  manager  could  adequately  control  the  test  configuration  of 
ADS  participants,  the  ADS  environment  data,  and  the  ADS  network  both  during  and  between  test 
events.  The  JADS  analysts  conducted  interviews  with  the  test  team  members  involved  in 
configuration  management  on  the  Phase  2  test.  The  JADS  analysts  also  monitored  the  progress 
of  test  and  network  configuration  from  formal  integrated  product  team  (IPT)  and  requirements 
meetings. 

Configuration  control  for  the  ETE  Test  synthetic  environment  proved  to  be  a  challenge.  The 
distributed  nature  of  the  test  made  configuration  control  more  complex  because  of  the  many 
different  organizations  involved  in  the  test.  Unlike  a  single-site  test  effort,  the  ETE  Test  involved 
more  than  half  a  dozen  organizations  and  two  branches  of  the  military.  The  added  difficulty  of  the 
distributed  network  made  fi:equent  technical  meetings  and  formal  IPTs  a  must.  These  meetings 
provided  the  test  manager  with  the  ability  to  track  progress  and  problems  with  the  network 
configuration  and  data  formats.  These  meetings  also  provided  the  forum  for  the  system  experts  to 
resolve  the  issues  with  all  those  who  were  affected. 
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This  process  proved  to  be  effective  in  controlling  the  Phase  2  configuration  during  the  test  build¬ 
up  and  preparation  phase.  In  addition,  there  were  no  configuration  control  problems  during  the 
OT  exectution. 

Configuration  control  of  an  ADS  test  can  add  significant  management  issues  when  compared  to 
non-ADS  efforts.  This  added  complexity  requires  a  higher  level  of  involvement  by  the  test 
manager,  as  well  as  more  frequent  face-to-face  meetings  among  technical  personnel  from  the 
various  ADS  test  nodes.  An  ADS  test  manager’s  job  would  be  eased  with  the  aid  of  an 
integration  engineer. 

4.2.3  JADS  Objective  2-3.  Develop  and  assess  methodologies  associated  with  ADS  for 
T&E. 

4.2.3. 1  JADS  Subobjective  2-3-2.  Develop  and  assess  methodologies  associated  with  test 
execution  and  control  for  tests  using  ADS. 

JADS  Measure  2-3-2-3.  Degree  to  which  protocols,  processes,  and  procedures  are  needed 
to  enable  effective  centralized  test  control. 

This  measure  determined  the  degree  to  which  protocols,  processes,  and  procedures  were  needed 
to  enable  effective  centralized  test  control  in  a  distributed  environment.  JADS  personnel  analyzed 
test  logs,  test  team  discussions,  and  after-action  reports  to  determine  how  well  the  test  control 
procedures  worked  and  how  they  could  be  improved. 

The  test  control  procedures  used  during  the  Phase  2  test  were  developed  and  refined  during  the 
functionality  and  integration  and  risk  reduction  tests.  These  procedures  included  standardized 
checklists  addressing  the  start-up  and  shutdown  of  the  ETE  Test  network  and  the  conduct  of  each 
test  trial.  The  network  checklist  included  procedures  to  verify  network  connectivity,  data  storage 
space,  and  time  server  accuracy  and  to  test  network  transmission  prior  to  test  start.  The  test 
conduct  checklist  included  detailed  start-up  procedures  for  each  test  node.  These  checklists  are 
provided  in  Appendix  A  to  this  report. 

Communication  procedures  were  also  developed  during  the  FI  and  risk  reduction  tests.  Based  on 
the  experience  gained  during  these  tests,  the  Phase  2  testers  were  able  to  use  the  conference 
phone  lines  to  effectively  resolve  system  failures  and  to  facilitate  useful  discussions  of  the  after¬ 
action  reports. 

Effective  centralized  test  control  was  exercised  through  the  test  controller  located  at  the  TCAC. 
The  test  controller  was  responsible  for  starting  and  stopping  each  trial,  monitoring  the  status  of  all 
nodes  and  network  links,  and  declaring  holds  and  restarts  when  necessary.  The  test  controller 
was  able  to  continuously  monitor  the  status  of  the  trials  through  a  combination  of  network 
monitors  in  the  TCAC  and  voice  communications  with  key  personnel  at  each  site.  These 
procedures  proved  effective  for  timely  detection  and  resolution  of  system/network  failures. 
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Test  control  in  a  distributed  computing  environment  offers  many  challenges  not  experienced  in 
conventional  testing.  The  designers  of  a  test  network  should  carefully  plan  for  test  control,  noting 
that  good  communication  is  required  among  all  network  nodes.  Otherwise,  poor  test  control  can 
result  in  inaccurate  and  untimely  information  being  disseminated  throughout  the  test  architecture 
and  can  waste  valuable  test  time  and  test  assets. 
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5.0  Lessons  Learned 


5.1  Technical  Lessons  Learned 

5.1.1  Simulations 

Simulations  when  used  in  ADS  testing  should  be  carefully  planned  and  developed.  The  ADS 
tester  must  also  be  in  close  communication  with  each  organization  modifying  simulations  crucial 
to  the  test.  The  ETE  Test  Phase  2  succeeded  because  of  the  relatively  high  degree  of  reliabihty  of 
the  simulations  used  in  the  test.  If  simulation  execution  problems  had  occurred,  the  ETE  Test 
team  had  arranged  for  a  quick  response  by  the  personnel  needed  to  fix  those  difficulties. 

5.1.2  Interfaces 

Distributed  testing  often  requires  linkage  among  dissimilar  facilities,  network  equipment,  and 
simulations.  However,  careful  planning  can  significantly  reduce  the  potential  for  difficulties 
arising  from  network  interface  problems.  As  part  of  their  planning,  the  ETE  Test  team  bought 
standard  network  equipment  for  aU  of  the  sites.  Thus,  the  configuration  of  the  ETE  Test 
environment  did  not  pose  any  problems  during  the  Phase  2  test. 

5.1.3  Networks 

-  Time  sources  must  be  synchronized.  Time  must  be  synchronized  from  the  same  time  source, 
then  validated  at  each  network  node  to  ensure  that  accurate,  synchronized  time  is  recorded  at 
each  node.  The  time  server  used  in  the  ETE  Test  worked  very  well,  ensuring  that  aU  loggers 
were  set  at  the  same  time  and  keeping  time  differences  between  loggers  to  a  minimum. 
Having  the  same  time  at  all  loggers  helped  speed  up  the  analysis  and  allowed  for  the  use  of 
automated  analysis  tools. 

Data  collection  methods  should  be  built  into  the  simulation.  The  ETE  Test  environment  failed 
to  take  data  collection  to  the  next  level.  Loggers  with  time  stamping  were  used  to  record 
PDUs  as  they  were  going  in  and  out  of  simulations.  However,  data  collected  from  within  the 
simulations  were  not  time  stamped  or  synchronized  with  the  logger  data.  This  presented 
considerable  problems  with  the  data  analysis  of  the  entire  C4ISR  system.  Much  of  this 
analysis  was  done  by  hand  or  accomplished  separately  from  the  JADS  logger  data. 

5.1.4  Instrumentation 

-  Special  equipment  was  necessary  for  ADS  network  check-out  and  verification.  Special  test 
equipment  and  networking  tools  wiU  rapidly  isolate  the  specific  cause  of  network  and 
ADS/DIS  problems.  Without  the  special  equipment,  troubleshooting  would  have  been 
accomplished  by  trial  and  error  increasing  time,  cost,  and  personnel.  In  addition,  the  key 
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Network  and  Engineering  personnel  should  be  trained  in  the  use  of  the  special  test  equipment 
and  networking  tools. 

-  The  flow  and  transfer  of  PDUs  were  critical  to  the  ETE  Test.  PDU  traffic  was  continuously 
monitored  at  all  nodes  to  ensure  that  PDUs  were  constantly  flowing.  Since  the  ETE  Test 
manning  levels  were  insufficient  to  enable  watching  these  monitors  at  aU  time,  problems  that 
interrupted  PDU  flow  were  not  always  immediately  noticed.  Some  type  of  audio  warning 
device  might  have  reduced  test  time  loss  from  a  few  minutes  to  seconds. 

5.2  Infrastructure  and  Process  Lessons  Learned 

5.2.1  Procedures 

5. 2. 1.1  Planning 

-  The  requirements  for  an  ADS  test  must  be  clearly  defined  early  in  the  test  planning  phase. 
Detailed  planning  and  coordination  are  required  to  ensure  a  common  understanding  of  aU 
requirements,  procedures,  and  test  objectives  since  individual  facilities  are  generally  unfamiliar 
with  conducting  coordinated,  distributed  T&E  tests. 

-  Flexibility  in  planning  is  essential.  When  doing  something  that  has  never  been  done  before, 
preconceived  notions  of  how  the  test  should  be  executed  will  have  to  change  as  more  is 
learned.  Be  open  to  new  ideas,  as  some  of  the  old  ideas  from  the  early  stages  of  an  ADS  test 
program  may  become  very  expensive  to  bring  to  fruition.  The  ETE  Test  Phase  2  was 
originally  slated  to  have  nine  scenarios.  As  the  requirements  for  each  scenario  increased,  their 
development  costs  also  grew.  These  added  costs  eventually  led  to  the  deletion  of  the  last  four 
scenarios. 

-  Plan  for  the  unexpected.  Halfway  through  the  ETE  Test  Phase  2  one  of  the  key  T-1  lines  was 
inadvertently  disconnected.  This  delayed  the  test  by  two  days  during  the  most  critical  portion 
of  the  test  while  technicians  restored  the  lines.  Travel  plans  had  to  be  changed  and  budgets 
were  strained.  If  possible,  plan  extra  days  on  the  end  of  a  test  period  that  can  be  eliminated  if 
all  goes  well.  It  is  much  easier  to  return  early  than  to  stay  longer. 

-  Minimize  the  impact  of  hardware  problems.  When  using  a  comphcated  ADS  network  with  a 
vast  array  of  equipment,  hardware  problems  wUl  occur.  Plan  in  such  a  way  that  unexpected 
hardware  problems  do  not  completely  disrupt  the  test.  During  the  ETE  Test  Phase  2,  steps 
were  taken  to  ensure  that  hardware  problems  did  not  disrupt  the  test  for  long  periods  of  time. 
For  example,  data  saves  were  accomplished  frequently.  In  addition,  the  network  was 
constantly  monitored  to  ensure  that  hardware  problems  were  fixed  as  soon  as  possible. 
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5.2. 1.2  Development 


-  Use  a  stepping  stone  approach  to  testing  where  each  successive  ADS  test  builds  on  the 
success  of  earher  tests.  This  “test,  analyze,  fix,  test”  approach,  in  concert  with  structured, 
independent  testing  of  the  network,  will  greatly  improve  the  chances  for  successful  ADS 
testing. 

-  Risk  reduction  testing  prior  to  actual  test  execution  will  help  test  team  personnel  identify  and 
resolve  potential  ADS  system  problems. 

-  Understand  communication  requirements.  Because  of  some  changes  in  the  test,  voice 
communication  requirements  between  the  Fort  Hood  node  and  the  White  Sands  node  were 
dramatically  increased.  The  only  way  for  those  nodes  to  communicate  was  through  the  direct 
line  to  the  TCAC.  This  tied  up  the  direct  line  for  extended  periods  of  time.  For  the  next  test, 
another  connection  wiU  be  added  for  direct  communications  between  Fort  Hood  and  White 
Sands. 

5. 2. 1.3  Execution 

-  Briefings  are  needed  before  and  after  each  ADS  test.  These  briefings  should  include  such 
information  as  the  test  objectives,  telephone  numbers  to  use  for  test  control,  the  test 
configuration  of  each  facility,  instrumentation  and  data  collection  requirements,  go/no  go 
criteria,  contingency  and  backup  plans,  and  test  conduct.  A  briefing  checklist  should  be 
developed  and  used. 

-  Test  control  procedures  should  be  well  rehearsed.  When  many  people  are  communicating  on 
one  phone  hne,  a  response  order  should  be  estabhshed  and  strictly  followed  to  save  valuable 
test  time. 

-  Take  advantage  of  the  opportunities  provided  by  ADS  technology.  At  the  beginning  of  Phase 
2,  the  ETE  Test  team  intended  to  log  aU  operator  actions  in  the  LGSM.  As  the  test 
progressed,  the  JADS  analysts  realized  their  efforts  to  obtain  data  from  the  LGSM  operators 
would  hamper  their  activities.  Making  use  of  the  increase  in  test  time  provided  by  the  ADS 
environment,  the  JADS  analysts  were  able  to  automate  most  of  their  LGSM  data  collection 
activities  and  reduce  the  impact  on  the  LGSM  operators. 

5. 2. 1.4  Evaluation 

-  Effective  data  management  is  needed.  Linked  facilities  can  generate  a  large  volume  of  data  at 
distributed  locations.  Without  careful  planning,  key  data  may  not  be  collected  and/or 
transmitted  to  the  analysis  center,  and  data  collected  at  the  network  nodes  may  not  be  in  a 
useful  form  for  centrahzed  analysis.  Before  ADS  testing,  a  comprehensive  data  management 
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plan  must  clearly  identify  the  data  to  be  collected  at  each  network  node,  onsite  processing  of 
the  data,  and  the  data  to  be  transmitted  to  the  analysis  center. 

-  Pretest  rehearsals  are  also  useful  for  improving  evaluation  procedures.  The  EXE  Test  team 
improved  its  data  collection  and  analysis  processes  as  a  result  of  experiences  from  the 
functionality  and  integration  and  risk  reduction  tests. 

5. 2. 1.5  Command  and  Control 

-  Have  test  controllers  who  are  extremely  famihar  with  the  test  and  network  configuration.  The 
Phase  2  test  succeeded  partly  because  it  had  an  experienced  test  controller  with  the  necessary 
tools  to  evaluate  problems  and  the  authority  to  make  meaningful  decisions  regarding  test 
problems. 

-  Have  a  centralized  test  control  center.  The  JADS  TCAC  is  configured  to  allow  for 
convenient,  instant  communications  with  aU  the  nodes.  It  acted  as  the  central  point  of  contact 
between  the  nodes  and  for  aU  problems.  The  test  controUer  kept  track  of  test  progress  and 
documented  any  problems  that  occurred. 

-  Estabhsh  control  over  resources.  Linking  various  facilities  using  ADS  can  require  significant 
development  of  facihty  interface  hardware  and  software.  A  weU-organized  approaeh  by 
experienced  personnel  enabled  the  ETE  Test  team  to  surmount  problems  encountered  at  Fort 
Hood,  the  most  comphcated  node  in  terms  of  getting  aU  necessary  hardware  estabUshed  and 
connected  before  the  Phase  2  test. 

-  Distributed  tests  require  personnel  distribution.  When  many  distributed  nodes  are  required  for 
the  successful  completion  of  a  test,  personnel  wiU  need  to  be  located  at  these  nodes.  The 
complexity  and  input  an  individual  node  contributes  should  guide  the  assignment  of  personnel. 
The  ETE  Test  Phase  2  required  several  people  at  the  Fort  Hood,  Northrop  Grumman  and 
TCAC  nodes;  only  one  person  was  needed  at  the  White  Sands  node.  The  Fort  SUl  node  used 
only  resident  personnel. 

5.2.2  Policy 

-  Network  management  and  troubleshooting  must  be  disciplined  and  organized  with  a  thorough 
understanding  and  strong  configuration  control  of  the  ADS  network.  This  approach  enabled 
the  ETE  Test  Phase  2  team  to  quickly  recover  from  initial  pretest  network  difficulties  and  to 
go  on  and  achieve  five  days  of  successful  execution  and  data  collection. 

-  Flexibility  is  also,  needed.  When  one  of  the  ETE  Test  network’s  T-1  fines  was  disconnected, 
JADS  personnel  were  able  to  quickly  develop  and  implement  a  contingency  plan.  Upon 
restoration  of  the  T-1  fine,  the  network  was  soon  returned  to  its  original  configuration  and  the 
test  continued. 
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5.2.3  Personnel 


-  Personnel  involved  in  a  distributed  test  should  understand  the  “big  picture.”  When  people  are 
geographically  separated,  it  becomes  easy  for  them  to  focus  on  their  own  individual  portion  of 
the  test.  When  problems  arise,  personnel  who  understand  the  entire  test  and  the  overall 
network  will  find  solutions  much  faster. 
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6.0  Conclusions/Recommendations 


6.1  Utility 

6.1.1  Utility  Conclusions 

6. 1.1.1.  Enhanced  Testing.  An  ADS  environment  can  enhance  the  testing  of  C4ISR  systems. 

-  The  Phase  2  test  showed  that  ADS  technology  can  realistically  represent  a  C4ISR  system 
enabling  the  system’s  users  to  collect  valid  and  useful  operational  information. 

-  Compared  to  conventional  methods,  an  ADS  environment  can  reahsticaUy  test  C4ISR 
systems: 

-  in  larger  battlespaces.  It’s  possible  to  connect  several  Janus  simulations  together  in  order 
to  multiply  the  simulated  battlespace. 

-  with  larger  numbers  of  ground-based  entities  at  a  much  lower  cost. 

-  with  more  control  over  the  specific  aspects  of  the  scenarios  being  tested.  Using  ADS,  the 
test  director  is  not  at  the  mercy  of  a  training  exercise  over  which  he/she  has  no  direction. 
Rather,  the  test  director  can  control  the  simulated  entities. 

-  for  longer  periods  of  time  enabling  increased  data  collection  and  the  ability  to  analyze  and 
improve  the  data  gathering  process. 

-  under  realistic  but  unsafe  conditions,  such  as  convoy  vehicles  driving  across  rugged  terrain 
under  wartime  conditions. 

-  By  allowing  the  simulation  of  large  battlespaces  with  large  numbers  of  entities,  ADS 
technology  provides  testers  with  greatly  expanded  capabilities  for  test  concept  and  design. 

-  Testers  can  use  ADS  to  save  time,  resources  and  test  personnel  man-hours  by  linking  several 
pieces  of  equipment  and/or  facilities  together  for  simultaneous  testing  instead  of  conducting 
individual  tests  at  different  locations. 

-  ADS  technology  provides  access  to  elements  participating  in  the  test  from  their  normal  work 
locations,  greatly  reducing  the  additional  operational  tempo  required  to  support  both  testing 
and  test  integration,  training  and  rehearsal. 
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6. 1.1.2  DT&E  Utility.  The  Phase  2  ADS  architecture  can  allow  the  use  of  VSTARS  for 
developmental  testing  of  the  JSTARS  subsystems,  other  than  the  radar  subsystems,  including 
software.  A  ground-based  ADS  environment  offers  several  benefits  for  DT&E. 

-  Multiple  repetitions  of  the  same  scenario  can  be  performed  without  competing  for  scarce  test 
aircraft  and  range  resources. 

-  The  use  of  ground-based  simulations  and  hardware  offers  the  potential  for  enormous  cost 
savings,  as  compared  to  a  live  test  with  the  aircraft. 

-  Any  conceivable  test  case  can  be  “flown”  in  the  laboratory  without  worrying  about  safety  or 
limited  assets  provided  the  appropriate  scenario  generator  is  available. 

-  Bad  software  can  be  quickly  discarded  and  new  software  could  be  tried  the  next  day. 

-  Most  importantly,  when  a  live  test  is  flown,  as  it  must  be,  the  testers  can  be  reasonably  sure 
that  they  will  get  the  maximum  value  from  the  flight  and  test  conditions. 

6. 1.1. 3  Improved  Opportunities  for  Training.  If  an  ADS  environment  is  developed  for  testing, 
the  same  environment  can  easily  be  modified  and  transitioned  to  the  training  world.  ADS 
technology  can  then  improve  opportunities  for  training  with  a  C4ISR  system. 

-  Conventional  training  can  be  limited  by  the  availability  of  those  assets  making  up  the  C4ISR 
system’s  operational  environment.  ADS  technology,  by  simulating  those  key  assets,  can 
provide  longer  periods  of  time  for  realistic  operation  of  the  C4ISR  system. 

-  C41SR  system  operators  can  take  advantage  of  the  additional  training  time  provided  by  ADS 
technology  to  confirm  current  tactics  and  to  test  “what  if’  scenarios  and  new  tactics. 

-  A  C4ISR  simulation  incorporated  into  an  ADS  environment  can  enhance  operator  training  by 
providing  useful  information  on  the  battlefield  surveillance  and  the  targeting  process.  It  can 
also  help  the  battle  manager  by  providing  a  high-level  picture  of  the  entire  battlefield  and 
aiding  in  the  effective  allocation  of  battle  resources. 

-  An  ADS  environment  can  provide  C4ISR  system  operators  with  the  opportunity  to  check  the 
interoperability  and  compatibility  of  their  equipment. 

-  ADS  simulations  can  help  C4ISR  system  operators  familiarize  themselves  with  the  maneuver 
tactics  of  foreign  armed  forces  -  valuable  experience  for  possible  future  deployments. 

-  C41SR  system  operators  can  train  in  an  operational  environment  with  multiple  assets  using  the 
capabilities  of  ADS  technology  to  tie  several  pieces  of  equipment  and/or  facilities  together  and 
to  simulate  large  numbers  of  fielded  vehicles  and  associated  personnel.  In  contrast. 
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conventional  training  would  provide  far  fewer  assets  in  the  operational  environment,  if  any  at 
all. 

6.1.2  Utility  Recommendations 

-  Large  exercises  could  use  the  ETE  Test  environment  to  virtually  augment  the  battlefield  with 
simulated  targets.  During  Phase  4,  this  capability  will  be  demonstrated  with  the  integration  of 
a  live  E-8C  Joint  STARS  aircraft  into  the  ETE  Test  ADS  environment. 

-  An  ADS  environment,  like  the  ETE  Test  environment,  is  flexible  enough  to  allow  for  further 
expansion  and  increased  opportunities  for  testing  C4ISR  systems.  The  Janus  battlespace  can 
be  expanded  as  required.  Increasing  the  number  of  LGSMs  or  CGSs  would  create  more 
realistic  targeting  capabilities.  By  adding  other  assets  to  the  environment,  such  as  an 
unmanned  aerial  vehicle  (UAV)  or  a  tactical  aircraft  simulator,  the  robustness  of  the 
environment  could  be  significantly  enhanced. 

6.2  Technical 

6.2.1  Technical  Conclusions 

-  The  ETE  Test  Phase  2  required  only  a  small  portion  of  the  available  bandwidth.  This 
indicates  that  a  much  higher  packet  rate  could  be  applied  to  ADS  testing  without  causing 
bandwidth  problems.  The  available  bandwidth  could  have  been  used  to  facilitate  a  higher 
heartbeat  by  sending  more  frequent  updates  of  entity  state  PDUs  from  Janus.  During  most  of 
the  Phase  2  test,  a  15-minute  heartbeat  was  used  during  the  first  hour,  followed  by  a  low  rate 
of  change-of-state  PDUs  after  the  first  hour.  During  the  last  day  of  testing,  the  change-of- 
state  PDU  rate  was  dramatically  increased.  The  future  ETE  Test  phases  will  incorporate  the 
higher  PDU  rate  to  take  advantage  of  the  available  bandwidth. 

-  The  ETE  Test  network  was  highly  reliable  and  stable  during  the  Phase  2  test  following  the  fix 
of  the  T-1  problems  experienced  on  28  and  29  September. 

-  The  ETE  Test  team’s  extensive  risk  reduction  testing  played  an  important  role  in  eliminating 
the  presence  of  major  ADS  system  failures  druing  Phase  2  testing.  Many  ADS  problems  were 
identified  and  resolved  during  this  risk  reduction  period,  thus  minimizing  their  potential  for 
affecting  the  actual  testing.  As  a  result,  the  ADS  problems  noted  in  Table  9  had  only  a  minor 
impact  on  Phase  2  testing  and  caused  only  brief  delays  or  reductions  in  capability. 

6.2.2  Technical  Recommendations 

-  With  careful  planning  and  resource  management,  testers  can  address  the  issues  associated  with 
integrating  simulations  into  an  ADS  test  environment. 
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-  Identify  the  assumptions  and  limitations  associated  with  those  simulations. 

-  Budget,  schedule,  and  provide  the  manpower  necessary  to  develop  the  simulations. 
Simulation  development  is  typically  labor  intensive  and  thus  costly. 

-  Determine  the  level  of  simulation  detail  needed  for  the  ADS  test.  Development  costs  are 
directly  related  to  the  level  of  simulation  detail. 

-  Identify  and  provide  training  for  the  users  of  the  simulations. 

6.3  Infrastructure 

6.3.1  Infrastructure  Conclusions 

-  ADS  can  reduce  the  number  of  troops  and  associated  equipment  involved  in  tests  because  of 
its  simulation  of  fielded  forces.  However,  the  ADS  infrastructure  requires  technical  persoimel 
to  set  up  and  execute  the  tests  and  to  analyze  the  test  results. 

-  Highly  structured  test  control  is  a  key  ingredient  for  ADS  test  success.  This  test  control 
should  include  formalized  procedures  with  an  emphasis  on  checklists. 

-  An  ADS  test  can’t  always  count  on  having  the  personnel  requirements  for  a  distant  node 
supplied  by  an  organization  local  to  the  node.  Even  if  an  ADS  test  is  able  to  employ  these 
people,  it  may  then  lose  them  to  other  activities  deemed  more  important  by  the  local 
organization. 

-  An  ADS  environment  necessitates  sophisticated  instrumentation  with  rigorous  processing 
speed,  data  storage,  and  data  integration  capabilities.  This  instrumentation  can  be  costly  and 
can  require  trained  personnel  for  its  successful  operation. 

-  Distributed  testing  typically  means  distributed  personnel  and  distributed  equipment. 
Distributed  personnel  lead  to  high  travel  costs.  Equipment  located  at  distant  network  nodes 
will  still  require  maintenance  either  through  contracts  or  trips  by  a  network  engineering  team. 

-  ADS  analysts  must  have  a  weU-planned  and  organized  approach  to  managing  the  large 
amounts  of  data  produced  from  ADS  testing. 

6.3.2  Infrastructure  Recommendations 

-  Make  every  effort  to  simplify  the  infrastructure.  Time  spent  in  the  planning  stages  of  an  ADS 
test,  with  an  emphasis  on  reducing  the  complexity  of  the  test  network,  is  time  well  spent.  Use 
proven  hardware  and  keep  it  the  same  wherever  possible. 
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Keep  in  mind  the  disadvantages,  as  well  as  benefits,  of  the  networked  nature  of  an  ADS 
environment.  The  ADS  tester  will  almost  always  be  dependent  upon  a  telecommunications 
provider.  For  example,  the  ETE  Test  environment  was  degraded  for  two  days  because  of  the 
inadvertent  termination  of  a  T-,1  line. 
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APPENDIX  A  -  Test  Procedures 


Al.O  Test  Procedures 

Various  types  of  checklists  were  used  during  the  execution  of  the  Phase  2  test.  The  Test  Control 
and  Analysis  Center  (TCAC)  test  controller  checklist  can  be  found  in  Section  Al.l,  TCAC  Test 
Procedures.  This  checklist  was  used  to  ensure  network  and  logger  functionality  and  to  provide 
overall  test  control  procedures.  Each  node  (White  Sands  Missile  Range  [WSMR],  Northrop 
Grumman,  and  Fort  Sill)  incorporated  the  logger  functions  from  the  TCAC  checklist  into  their 
own  checklist. 

Other  checklists  were  used  to  direct  the  operation  of  various  pieces  of  test  equipment.  An 
example  is  included  in  Section  A1.2,  TCAC  Plan  View  Display  (PVD)  Procedures. 

Section  A1.3,  WSMR  Procedures  is  representative  of  the  site-specific  checklists.  WSMR, 
Northrop  Grumman  and  Fort  SiU  all  developed  procedures  for  operation  of  the  End-to-End 
(ETE)  Test  environment  equipment.  Only  Fort  Hood,  the  only  site  without  a  logger,  failed  to 
develop  written  procedures.  Their  procedures  were  primarily  accomplished  by  the  resident 
specialists  who  have  their  own  procedures. 

Al.l  TCAC  Test  Procedures 

The  following  are  the  written  test  procedures  used  in  the  TCAC  during  Phase  2  testing. 

72  HOURS  PRIOR  TO  TEST 

Network  Coordinator: _ 

Date: _ Test  Time: _ to _ 

1.  _  Check  supplies. 

2.  Turn  on  equipment. 

_  a.  Turn  on  3  Barcos  (Spectrum  [Sun5]  on  1,  Janus  [hp735]  on  2,  and  NetVis  [indigo2]  on  3). 

b.  Log  in  as  “root”  to  indigo2  in  the  TCAC,  and  indy4  in  communications  room  1. 

_  1)  From  the  toolchest,  select  Toolbox,  JADS  Toolbox,  Monitor,  PDU  Monitor,  PDU 

Statistics,  Show  Stats  to  display  protocol  data  units  (PDUs). 

_  2)  From  the  toolchest,  select  NetVis,  NetGraph-ETE  to  display  network  traffic. 

_  3)  From  the  toolchest,  select  NetTests,  Status  Check  ETE  to  start  and  display  network 

connectivity  tests,  (uts  in  comm  rm  1  pings  wsmr,  ftsill,  and  fthoodafatads.  indigo2, 
pings,  grumman,  indy3,  and  sparcS  at  Ft  Hood). 

_  c.  In  the  TCAC,  mn  Spectrum  on  the  Sun20  (server)  and  SunS  (graph)  to  display  Zulu  time  and 

router  status. 

_  d.  Create  an  empty  file  “touch  /scripts  /  .  go”  in  grumman,  indigo2,  and  indy4. 
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3. 


Clear  router  interfaces.  To  clear  the  grumman_router,  jads_router,  and  fthood_router  from 
indigo2;  and  fthood_router,  ftsill_router,  and  wsmr_router  from  indy4,  run: 
“/scripts/clear_router  ete.” 

4.  _ Not  used. 

5.  Time  accuracy.  Verify  that  each  site  has  network  time  protocol  (NTP)  running. 

_  a.  From  uts,  run  “/scripts/check_time”  and  verify  that  the  offsets  for  ftsill  and  wsmr 

are  less  than  1  millisecond  (ms). 

_ b.  From  indigo2,  rlogin  to  indyl,  run  “/scripts/check_time .  ”  Verify  offsets  for 

grumman,  indy3,  and  sparcS  are  less  than  1  ms. 

6.  Available  disk  space.  Verify  that  each  logger  has  at  least  600  megabytes  (MB)  of  unused  disk 
space  available  on  the  /disk2  partition. 

_  a.  From  uts,  rlogin  to  ftsill  and  wsmr,  in  turn,  and 

from  indigo2,  rlogin  to  grumman,  and  indy3,  in  turn. 

_  b.  Run  “df  -k”  on  each  machine  (including  uts)  to  display  the  available  disk  space.  Verify  that 

each  has  at  least  600  MB  available. 

7.  Port  settings.  Verify  that  each  logger  is  set  to  port  3000  and  the  exercise  identification  (ID)  is  0. 
_  a.  From  uts,  rlogin  to  ftsill  and  wsmr,  in  turn,  and 

from  indigo2,  rlogin  to  grumman,  and  indy3,  in  turn. 

_  b.  Run  “more  /scripts/dt_logger”  to  view  the  file.  Look  for  the  entry: 

“ /usr/ local /bin/ jads_logger  3000  0  /disk2/logf iles 
/$testdate"_test"$testnum"_"$runnum"_"$site" .  log"  ”  entry  in 
two  places. 

8.  _ Voice  conference  net.  Verify  the  net  is  functional  by  dialing  in  from  two  different  phones  in  the 

TCAC  at  the  same  time  to  establish  the  net. 

9.  _  Not  used. 

10.  Data  collection  test  a: 

_  a.  From  uts,  rlogin  to  ftsill  and  wsmr,  simultaneously,  and 

from  indigo2,  rlogin  to  grumman,  and  indy3,  simultaneously. 

_ b.  Start  the  ftsill,  grumman,  indy3,and  uts  loggers  using  test  number  “0  00”  and  run  number 

“a”  (i.e.  -  “/scripts/dt_logger  000  a”). 

_  c.  Run  the  “/scripts/run_player  3000  /disk2/logf  iles/ne_test .  log” 

file  on  the  wsmr  machine. 

_  d.  Determine  when  mn  is  complete.  Stop  all  loggers  (“Ctrl-C”). 

_  e.  Check  digital  communications  terminal  (DCT)  results.  Verify  reception  of  2281  PDUs  on 

grumman,  indy3,  and  uts  (or  indy4)  loggers.  (No  PDUs  at  ftsill). 

11.  Data  collection  test  b: 

_ a.  From  uts,  rlogin  to  ftsill  and  wsmr,  simultaneously,  and 

from  indigo2,  rlogin  to  grumman,  and  indy3,  simultaneously. 

_ b.  Start  the  grumman,  indy3,  uts  and  wsmr  loggers  using  test  number  “000”  and  run  number 

“a”  (i.e.  -  “/scripts /dt_logger  000  a”). 
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_ _  c.  Run  the  “/scripts/run_player  3000  /disk2/logfiles/ne_test  .log” 

file  on  the  ftsill  machine. 

_  d.  Determine  when  run  is  complete.  Stop  all  loggers  (“Ctrl -C”). 

_  e.  Check  DCT  results.  Verify  reception  of  2281  PDUs  on  grumman,  indy3,  and  uts  loggers. 

12.  _ Report  the  results  of  the  network  checks  to  the  test  controller.  Supervise  repairs  as  necessary  to 

prepare  equipment  for  the  test  sequence. 


PRETEST  (DAY  OF  TEST) 

Network  Coordinator: _ 

Date: _ Test  Time: _ to _ 

1 .  _ Check  supplies.  Provide  checklists,  blank  log  sheets,  file  name  lists,  pens,  pencils,  scratch  paper, 

and  4  millimeter  (mm)  tape  cartridges  for  the  test. 

2.  Turn  on  equipment. 

_  a.  Turn  on  3  Barcos  (Spectrum  [Sun5]  on  1,  Janus  [hp735]  on  2,  and  NetVis  [indigo2]  on  3). 

b.  Log  in  as  “root”  to  indigo2  in  the  TCAC,  and  indy4  in  communications  room  1. 

_  1)  From  the  toolchest,  select  Toolbox,  JADS  Toolbox,  Monitor,  PDU  Monitor,  PDU 

Statistics,  Show  Stats  to  display  PDUs. 

_  2)  From  the  toolchest,  select  NetVis,  NetGraph-ETE  to  display  network  traffic. 

_  3)  From  the  toolchest,  select  NetTests,  Status  Check  ETE  to  start  and  display  network 

connectivity  tests,  (uts  in  Comm  Rm  1  pings  wsmr,  ftsill,  and  fthoodafatads.  indigo2, 
pings,  grununan,  indy3,  and  sparcS  at  Ft  Hood). 

_  c.  In  the  TCAC,  run  Spectrum  on  the  Sun20  (server)  and  SunS  (graph)  to  display  Zulu  time  and 

router  status. 

3.  Clear  router  interfaces.  To  clear  the  grumman_router,  jads_router,  and  fthood_router  from 
indigo2;  and  fthood_router,  ftsill_router,  and  wsmr_router  from  indy4,  run: 

_  “/scripts/clear_router  ete.” 

4.  _  Not  used. 

5.  Time  accuracy.  Verify  that  each  site  has  NTP  running. 

_  a.  From  uts,  run  “/scripts/check_time”  and  verify  that  the  offsets  for  ftsill  and  wsmr 

are  less  than  1  ms. 

_ b.  From  indigo2,  rlogin  to  indyl,  run  “/ script  s  /  check_t  ime .”  Verify  offsets  for 

grumman,  indy3,  and  sparcS  are  less  than  1  ms. 

6.  _  Available  disk  space.  Performed  at  each  logger  by  the  logger  operator. 

7.  _ Port  settings.  Performed  at  each  logger  by  the  logger  operator. 
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8.  _  Join  the  voice  conference  net.  Both  the  test  controller  and  the  network  coordinator  (NC)  dial 

61143  in  the  TCAC  to  establish  the  conference  net. 

9.  _  Time  synchronization,  ftsill,  grumman,  indy3,  and  wsmr  operators  check  global  positioning 

system  (GPS)  time  reception  by  typing  “date”  and  press  Enter  on  the  NC’s  mark.  Report  time  to 
NC. 

(NOTE:  indyl  is  time  server  for  classified,  uts  is  time  server  for  unclassified.) 

10.  Data  collection  test  a: 

_  a.  Cue  ftsill,  grumman,  mdy3,and  uts  operators  to  start  loggers  using  test  number  “000”  and 

run  number  “a”  (i.e.  -  “/ scripts /dt_logger  000  a”). 

_  b.  Cue  wsmr  Operator  to  run  “/scripts/run_player  3000 

/disk2/logf  iles/ne_test .  log”  file. 

_  c.  Determine  when  run  is  complete.  Cue  all  operators  to  stop  loggers  (“Ctrl-C”). 

_  d.  Check  DCT  results  -  Have  grumman,  indy3,  and  uts  operators  verify  reception  of  2281 

PDUs.  (No  PDUs  at  ftsill). 

11.  Data  collection  test  b: 

_  a.  Cue  grumman,  indy3,  uts,  and  wsmr  operators  to  start  loggers  using  test  number  “000”  and 

run  number  “b”  (i.e.  -  “/scripts/dt_logger  000  b”). 

_  b.  Cue  ftsill  operator  to  run  “/scripts/run_player  3000 

/disk2/logflles/ne_test .  log”  file. 

_  c.  Determine  when  run  is  complete.  Cue  all  operators  to  stop  loggers  (“Ctrl-C”). 

_  d.  Check  DCT  results  -  Have  grumman,  indy3,  uts,  and  wsmr  operators  verify  reception  of 

2281  PDUs.  (No  PDUs  at  ftsill). 

12.  _  Report  the  results  of  the  network  checks  (items  9-1 1)  to  the  test  controller. 


The  pretest  phase  is  now  complete.  Proceed  to  the  test  run  phase. 

NOTE:  Sometimes  the  logger  process  does  not  terminate  on  the  grumman  logger.  In  that  case,  run 
/  script  s  /  f  ind_logger  on  the  grumman  logger  to  kill  the  process  and  delete  the  old  logfile  before 
restarting  the  logger  with  the  same  filename. 


TEST  RUN 

Network  Coordinator: _ 

Date: _  Lab  Time: _ to _ 

1 .  _  Start  loggers.  Obtain  the  test  and  run  numbers  from  the  test  controller  and  record  on  the  log 

sheet.  Operators  are  cued  by  the  test  controller  when  to  start  loggers.  Record  start  time  on  the 
log  sheet. 

_ _  a.  Early  in  the  test  run,  verify  with  operators  that  all  loggers  are  receiving  PDUs  (number  is 

increasing. 

_ b.  Periodically  check  with  operators  that  all  loggers  continue  to  receive  PDUs  (number  is 

increasing). 
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_  c.  Periodically  check  “bat”  phone  operation  if  not  used  regularly. 

-  d.  Every  Vz  hour,  run  a  time  accuracy  check.  From  uts,  run  “/ script s /check_time”  to 

check  ftsill  and  wsmr.  From  indigo2,  rlogin  to  indyl  and  run  “/scripts/check_time”  to 
check  indyl,  gnimman,  and  tcacindy.  Time  offsets  should  be  <1  ms. 

_ e.  Keep  written  event  log. 

2-  Stop  loggers.  Loggers  stop  recording  data  when  directed  by  the  test  controller  (“Ctrl-C”). 

_  a.  Record  the  stop  time  and  the  total  number  of  PDUs  from  each  logger  on  log  sheet. 

- b.  Confirm  that  the  required  data  have  been  logged.  From  uts,  rlogin  to  ftsill  and  wsmr  and 

run  “Is  -1  /disk2/logfiles.”  From  indigo2,  rlogin  to  indy3,  and  grumman  and 
run  “Is  -1  /disk2/logfiles.”  Check  the  file  sizes;  the  filename  is 
“mmddyy_test#-run#_loggemame\og” 

3-  -  Subsequent  runs.  When  additional  runs  are  required,  repeat  steps  1  and  2  for  each  run. 


The  test  run  phase  is  now  complete.  Proceed  to  the  post-test  phase. 


POST  TEST  (DAY  OF  TEST). 

Network  Coordinator:  _ _ 

Date: _  Test  Time; _ to _ 

L  Remote  file  capture.  Consolidate,  compress,  and  copy  the  test  run  logger  files  from  each  remote 

site. 

- a.  For  classified  data,  rlogin  to  each  logger  (grumman  and  indy3),  in  turn,  from  tcacindy  in  the 

TCAC,  or 

For  unclassified  data,  rlogin  to  each  logger  (ftsill,  wsmr,  and  uts),  in  turn,  from  uts. 

- b.  If  only  1  file  for  the  day  exists  in  the  logger  at  a  site,  skip  to  step  c.  If  more  than  1  log  file  for 

the  day  exists  at  a  site,  consolidate  them  by  using  the  command 

“tar  cvf  imttddyy_sltenaine.  log.  t.a.x  iimddyy*  .log” 

where  *  is  the  wildcard  character  that  includes  all  the  files  for  that  day  for  that  site  name. 

(e.g.,  -  “tar  cvf  040798_wsmr.log. tar  040798*  .  log”  ). 

- c.  Compress  the  single  log  file  (e.g.,  -  “compress  04079 8_wsmr .  log”)  or  the  tar  file 

from  stepb  (“compress  040798_wsmr .  tar”). 

- d.  On  tcacindy,  run  “/ script s/rcp_etefile”  to  copy  the  tar'd  and  compressed  classified 

files  i“mmddyy_sitename.log.Z”)  from  both  grumman  and  indy3  loggers  to 
tcacindy  MiskZ/etdmmddyy/. 

- e.  On  uts,  run  “/ scripts/rcp_etef  ile”  to  copy  the  tar'd  and  compressed  unclassified 

files  Cmmddyy_sitename.log.z”)  from  ftsill,  uts,  and  wsmr  loggers  to 
-  uts:/disk2/ete//M//iddyy/. 

- f.  Copy  the  unclassified  files  from  step  e  to  4mm  tape  and  activate  the  write  protect  feature. 

g.  Move  the  tape  to  tcacindy  and  copy  the  unclassified  files  from  the  tape  to  IdiskZletelmmddyyl 
(i.e.,  from  the  etc  directory,  run  tar  xv  to  extract  the  files  from  the  tape  to  the  hard  drive). 

Make  sure  the  write  protect  feature  is  ON. 

2.  Backup  tapes. 
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_  a.  Create  a  backup  tape  of  the  files  in  tcaclndy:/disk2/ete/»i/Mf?J3'j/  using  either  the  “  tar  cv 

aanddyy"  command  while  in  the  ete  directory  (or  the  tape  tool  on  the  tcacindy  desktop), 
b.  Verify  the  backup  using  either  the  “tar  tv”  command  or  the  tape  tool. 

_  c.  Remove  the  tape  from  the  drive  and  label  it. 

_ d.  Repeat  a,  b,  and  c  to  create  a  duplicate  tape. 

_ e.  Deliver  both  tapes  to  the  ETE  Test  team  representative. 

3.  _ Delete  the  data  collection  test  and  the  backed-up  log  files  from  /disk2/logfiles/  on  all  loggers. 

4.  _  On  the  last  day  of  testing,  delete  the  file  “/scripts/ .  go”  in  grumman,  indigo2,  and  indy4. 

5.  _  Logoff  from  logger.  Turn  Off  the  monitor,  but  leave  the  central  processing  unit  (CPU)  On! ! ! 

6.  _ Participate  in  mission  debrief,  if  applicable. 


A1.2  TCAC  Plan  View  Display  (PVD)  Procedures 

The  following  procedures  were  used  to  initiate  test  monitoring  with  the  Janus  plan  view  display 
program.  This  is  representative  of  the  specific  checklists  developed  to  aid  in  the  operation  of  test 
equipment. 


Functionality/Integration  Test  Checklist 
(TCAC-PVD) 


Date:  _ 

Scenario:  _ 

Test  Start  Time  (z):  _  Test  Stop  Time  (z): 

Scenario  Start  Time:  _  Scenario  Stop  Time; 


Step  # 

mn 

Action 

Event 

Run  PVD  1 

1 

ETE 

Power  on  the  hp735  monitor. 

Log  on  to  the  hp735  as  hovey. 

2 

ETE 

From  the  xterm  window  that 

Use  this  alias  to  start 

appears,  type  pvd,  and  hit  enter. 

Janus  plan  view  display. 

ETE 

From  the  Janus  plan  view 

display  menu,  verify  the 

parameters  for  the  run: 
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Workstation  1 

Terrain  File _ 

Screen  File  _ 

Symbol  File  3 
Symbol  Size  10 
Terrain  File  Meridian  45 
Exercise  ID  BLANK 
Map  Spheroid  1 
Mode  BLANK 
Terminate  this  Run  N 

and  hit  keypad  enter. 


Wait  until  the  PVD  terrain  and 
combat  systems  databases  are 
loaded. 


Double  click  the 
Analyst_Workstation_WSl  icon 
to  bring  up  the  scenario 
window. 


From  the 

Analyst_Workstation_WSl 
scenario  window,  functions 


_ left  click  Draw  CAC. _ 

ETE  From  the 

Analyst_Workstation_WSl 
scenario  window,  CAC  File 
menu,  select  the  CAC  file 
number  to  display. 

Left  click  increases  number,  and 
right  click  decreases  number. 
Left  click  Add  to  display  the 
CAC. 


ETE  From  the 

Analyst_Workstation_WSl 
scenario  window,  function 
menu, 

_ left  click  Display. _ 

ETE  From  the 

Analyst_Workstation_WSl 
scenario  window  menu: 

Left  click  any  tick  on  the  zoom 
in/out  menu,  then  select  the 


Use  the  correct  terrain  file 
and  screen  file  for  the 
vignette. 


. . . 


Last  message:  Opening 
file 

../jads_ete/trn/TSCRN _ . 

DAT 


Places  command  and 
control  overlays  on  the 
scenario  box. 


Ready  to  receive  and 
display  DIS  PDUs. 


Used  as  necessary  to 
zoom  in/out  of  the 
scenario  box. 
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desired  zoom  point  on  the 
scenario  box. 

Used  as  necessary  to  add 

Left  click  CAC. 

or  remove  the  command 
and  control  overlays 
which  have  been  added  in 
step  7. 

Left  click  Display. 

Used  as  necessary  to  start 
or  stop  Janus  plan  view 
display  from  receiving 
PDUs. 

Left  click  Clear. 

Used  as  necessary  to  clear 
any  text  or  information 
displayed  on  the  scenario 
box. 

NOTE:  A  particular 
function  is  active  when 
highlighted. 

Step# 

POC 

Action 

Event,  ■;yi 

Go/No 

Go 

Stop  PVD  1 

1 

ETE 

From  the 

Analyst_Workstation_WSl 
scenario  window, 
right  click  End. 

Shuts  down  PVD. 

2 

ETE 

Minimize  the 

Analyst_Workstation_WSl 
scenario  window. 

In  the  xterm  that  remains, 
verify  this  message: 

STOP - JANPVD 

Program  Terminated - 

3 

ETE 

From  the 

Analyst_Workstation_WSl 

icon, 

right  click  and  choose  close. 

Closes  the  scenario 
window. 

Step  # 

Action 

.. 

^1%'  Go/No  ^ 
aI:  Gi^.:  - 

Shut  Down  Test  I 

1 

ETE 

Left  click  EXIT  from  the  HP 
VUE  front  panel. 

Signs  off  the  hp735. 
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2 

ETE 

Left  chck  Continue  logout  from 
the  dialog  box. 

Confirms  desire  to  log 
out. 

3 

ETE 

Power  off  the  hp735  monitor. 

A1.3  WSMR  Procedures 

This  checklist  is  representative  of  the  individual  site  checklists.  It  incorporates  the  logger 
functionality  and  the  site  specific  actions  required  by  the  operator(s).  These  are  maintained  by  the 
site  specialists  and  updated  as  changes  are  required. 

Functionality /Integration  Test  Checklist 
(WSMR) 


Date:  _ 

Scenario:  _ 

Janus  File:  _ 

Indy  File:  _ 

Test  Stop  Time  (z): 
Scenario  Stop  Time: 


Test  Start  Time  (z): 
Scenario  Start  Time: 


IjjSliB  # 

POC 

'  ■ 

L _ _ 

Action 

Event 

Go/No 
Mo  X 

Network  Activation  | 

1 

ETE 

and 

N&E 

Verify  operation  of  hotlink 
phone.  If  no  go,  contact  N&E 
to  fix  the  network. 

Enables  secure  and 
unclassified  voice 
communications. 

2 

ETE 

and 

WSM 

R 

Verify  that  WSMR  indy  and  the 
WSMR  hp715  are  on  the  JADS 
ETE  network. 

Initial  step  in  ensuring 
network  is  operational. 

f 

3 

N&E 

Verity  N&E  has  cleared  and 

reset  routers. 

Clears  router  interface 
cards. 

. 

.r  '#/•  :■  ■ 

4 

ETE 

Power  on  the  WSMR  monitor. 
Log  on  to  WSMR  as  dislog. 
From  a  Unix  shell  window  as 
su,  run  ! scripts/restart. 

Restarts  the  indy. 

3  ^  :^3 

5 

ETE 

After  a  successful  restart,  log 
on  to  wsmr  as  dislog. 

Signon  is  used  for 
checking  network 
communications  and 
logging  PDU  data. 

.  ■  ■  .  ; 
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6 

ETE 

From  a  Unix  shell  window  as 

Verifies  that  each  network 

su, 

link  is  operational.  3% 

run  /scripts/pingjtest  to  get 

loss  at  Fort  Sill  and  uts  is 

ping  statistics  for  each  remote 
site. 

normal. 
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7 

ETE 

From  a  Unix  shell  window  as 
su,  run  /scripts/check_time. 

Displays  the  offset  from 
uts.  Should  be  less  than  1 

ms. 

8 

ETE 

From  a  Unix  shell  window  as  su 
and  at  the  test  controller’s 
direction, 

run  /scripts/run _j)layer  3000 
/disk2/logfiles/ne_test.log  to 
check  ability  to  send  PDUs  to 
each  remote  site. 

Verifies  sending  2281 

PDUs  and  receiving  the 
same  number  of  PDUs  at 
each  remote  site. 

. 

fe,::;.,,:;:. . . . 

9 

ETE 

From  a  Unix  shell  window  as  su 
and  at  the  test  controller’s 
direction, 

mn/scripts/dt  logger 

to  check  ability  to 
receive  PDUs  from  a  remote 
site.  At  the  test  controller’s 
direction,  hit  Ctrl-C  to  end  the 
logfile. 

Verifies  receiving  2281 
PDUs  from  a  remote  site. 

-  ■  a  r  = 

10 

ETE 

From  a  Unix  shell  window, 
cd  /usr/local/bin  and 
run  ./display _pdu_rate. 

Select  port  3000  0. 

Left  click  start. 

Verifies  that  PDU_rate  = 

0.  Ensures  that  there 
aren’t  any  DIS 
communications  before 
the  start  of  testing. 
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Step# 

POC 

Action 

Event 

Go/No 

Go 

Start  WSMR  Logger 

ETE 

From  a  Unix  shell  window  as  su 
on  WSMR,  run 
/scripts/dt  looser 

Script  that  runs  the  JADS 
logger. 

ETE 

Verify  the  logfile  name  as 
/disklAo^files/ 
mr.log  and  port  3000. 

Opens  port  3000  to  listen 
and  log  all  DIS 
communications.  Writes 
to  the  listed  logfile. 

;3eg,g 

POC 

Action 

Event 

Go/No 

Go 

Start  Janus 

1 

ETE 

Power  on  the  cl 80  monitor. 

Log  on  to  the  cl 80  as  JADS. 

2 

ETE 

From  an  hpterm  window,  type 
janus.exe,  and  hit  enter. 

3 

ETE 

Left  click  PE  (Program 
Execution)  from  the  Janus  User 
Options  menu. 

4 

ETE 

Left  click  JE  (Janus  Execution) 
from  the  Program  Execution 
menu. 

5 

ETE 

Type  desired  scenario  number 
for  the  run.  and  hit 

enter. 

Type  run  number  1,  and  hit 
enter. 

6 

ETE 

Hit  enter  again  to  continue. 

7 

ETE 

Verify  that  1  is  entered.  Hit 
enter  one  more  time. 

8 

ETE 

From  the  Janus  Runtime 

Screens  menu, 
left  click  11. 

Verify  time  of  day  is  correct  for 
the  vignette,  and  hit  keypad 
enter. 

Use  this  executable  to 
start  Janus. 


Brings  up  the  Program 
Execution  menu. 


First  step  in  defining  the 
scenario. 


Tells  Janus  which  scenario 
to  run. 


Ready  to  continue. 


Use  a  normal  run. 


Verifies  time  of  day. 


EXE  From  the  Janus  Runtime 
Screens  menu, 
left  click  22. 

Verify  that  there  is  a  setup  for: 
1X5  Number  i,and  Side  1, 
and  hit  keypad  enter. 


EXE  From  the  Janus  Runtime 
Screens  menu, 
left  click  66. 

Verify  the  DIS  operational 
parameters  for  the  run: 

Janus  side  i 
DIS  side  2 
DIS  COMM  calls/sec 

Units  processed/COMM  call 

Xerrain  File  Meridian  (+E)  45 
Heartbeat(s) 


Verifies  that  a  controller 
workstation  has  been 
configured. 


Verifies  DIS  parameters. 
Calculate  the  new 
heartbeat  as  follows: 

CxRxH<X 

where 

C  =  calls/sec, 

R  =  units/call, 

H  =  heartbeat,  and 
X  =  total  number  of  units 
in  scenario 


Dead  Reckoning  Xhreshold 
999 

Site  TRAC-WSMR  23 
Host  CPU  HP  4 
Exercise  JADS-ETE  4 
DIS  version  transmit  4 
DIS  version  receive  4 


_ and  hit  keypad  enter. _ 


EXE  Left  click  JJ  (Begin  Janus)  Loads  the  Janus  scenario, 

from  the  Janus  Runtime  Screens 
menu. 


Wait  until  the  Janus  scenario 
loads.  Verify: 

Scenario  number  _ 

Xotal  number  of  units 


Double  click  the  sidel  icon  to 
bring  up  the  scenario  window. 


Xhis  brings  up  the 
scenario  window  which 
allows  a  Janus  operator  to 
interact  (game)  the 


exercise. 


76 


Step  #  P 


Action 


Run  Scenario 


From  the  sidel  scenario 
window, 

left  click  Z)/5. _ _ 

ETE  Erom  the  sidel  scenario 
window, 

left  click  SJARr. 


ETE  Minimize  the  Janus  scenario 

window  (sidel).  Type  rr  in  the 
Janus  window,  and  hit  enter. 


ETE  Type  n  and  hit  enter. 


5  ETE  Hit  enter  again. 


6  ETE  Double  click  the  Janus  scenario 
window  (sidel). 


ETE  Verify  that  loggers  are  logging. 


Event 


DIS  button  highlights. 
Opens  DIS 
communications. 


First  step  in  running  a 
Janus  scenario. 


Ready  to  continue  the 
Janus  run. 


No  planned  save. 


Default  checkpoint 
frequency. 


Verifies  scenario 
movements  and  a  running 
time  of  day  counter. 


Go/No 

Go 


POC 

Action 

Event 

1"- 

Go/No 

Go 

Stop  Scenario 

1 

ETE 

From  the  sidel  scenario 

window, 

left  chckZJ/S. 

DIS  button  unhighlights. 
Closes  DIS 
communications. 

From  the  sidel  scenario 
window, 

right  click  ADM/V. 


ETE  Left  click  EJ  (End  J anus) . 


ETE  Right  click  2  times. 


Brings  up  options  menu. 


Quits  the  scenario  run. 


Completely  closes  Janus. 


5  ETE  Left  click  EXIT  from  the  HP  Sign  off  the  hp7 15. 

VUE  front  panel. 


Step# 

Action 

Event 

Go/No 

Go 

Shut  Down  Test 

1 

ETE 

Power  off  the  cl 80  monitor, 
and  shutdown  CPU. 

2 

ETE 

Make  sure  that  JADS  N&E 

Ensures  data  integrity. 

and 

FTP 

This  file  will  be  analyzed 

N&E 

/diskl/loefiles/  ws 

mr.log 

back  to  JADS  and  place  the  file 
in 

/usr/testdata2/logs/ete/DDMM 

YY 

by  JADS  analysts. 

3 

ETE 

Power  off  the  wsmr  monitor. 

mm 

4 

ALL 

After-action  review 
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Appendix  B 
Pages  79-86 
intentionally  removed 


APPENDIX  C  -  Glossary 


A 

Accreditation.  See:  distributed  simulation  accreditation,  model/simulation  accreditation. 

Accuracy.  The  degree  of  exactness  of  a  model  or  simulation  relative  to  an  established  standard 
with  high  accuracy  implying  low  error.  [DIS] 

Activity.  An  event  that  consumes  time  and  resources  and  whose  performance  is  necessary  for  a 
system  to  move  from  one  event  to  the  next.  [DIS] 

Advanced  Distributed  Simulation  (ADS).  A  set  of  disparate  models  or  simulations  operating  in 
a  common  synthetic  environment.  The  ADS  may  be  composed  of  three  modes  of  simulation: 
live,  virtual  and  constructive,  where  the  latter  can  be  seamlessly  integrated  within  a  single 
exercise.  See  also:  live  simulation;  virtual  simulation;  constructive  simulation.  [DIS] 

Aggregate.  An  activity  that  combines  individual  entities  into  a  singular  entity.  Contrast  with: 
disaggregate.  [DIS] 

B 

Battlespace.  The  three-dimensional  battlefield.  [DIS] 

Benchmark,  (v)  The  activity  of  comparing  the  results  of  a  model  or  simulation  with  an  accepted 
representation  of  the  process  being  modeled,  (n)  The  accepted  representation  of  the  modeled 
process.  [DIS] 

Bit.  The  smallest  unit  of  information  in  the  binary  system  of  notation.  [IEEE  1278.1] 

Broadcast.  A  transmission  mode  in  which  a  single  message  is  sent  to  all  network  destinations, 
i.e.,  one-to-all.  Broadcast  is  a  special  case  of  multicast.  Contrast  with:  multicast;  unicast. 
[IEEE  1278.2] 

c 

Compatible.  Two  or  more  simulations  are  DIS  compatible  if  (1)  they  are  DIS  compliant,  and  (2) 
their  models  and  data  that  send  and  interpret  PDUs  support  the  realization  of  a  common 
operational  environment  among  the  systems  (coherent  in  time  and  space).  Contrast  with: 
compliant,  interoperable.  [DIS] 

Compliant.  A  simulation  is  DIS  compliant  if  it  can  send  or  receive  PDUs  in  accordance  with 
IEEE  Standard  1278  and  1278  (working  drafts).  A  specific  statement  must  be  made 
regarding  the  qualifications  of  each  PDU.  Contrast  with:  compatible,  interoperable.  [DIS] 

Conceptual  Model.  A  description  of  the  content  and  internal  representations  which  are  the  user's 
and  developer's  combined  concepts  of  the  exercise.  It  includes  logic  and  algorithms  and 
explicitly  recognizes  assumptions  and  limitations.  [DIS] 

Constructive  Simulation.  Models  and  simulations  that  involve  simulated  people  operating 
simulated  systems.  See  Also:  war  games;  higher  order  model  (HOM).  [DIS] 

Continuous  Model.  (1)  A  mathematical  or  computational  model  whose  output  variables  change 
in  a  continuous  manner;  that  is,  in  changing  from  one  value  to  another,  a  variable  can  take  on 
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all  intermediate  values.  For  example,  a  model  depicting  the  rate  of  air  flow  over  an  airplane 
wing.  Syn:  continuous-variable  model.  (2)  A  model  of  a  system  that  behaves  in  a  continuous 
manner.  Contrast  with:  discrete  model.  [DIS] 

Continuous  Simulation.  A  simulation  that  uses  a  continuous  model.  [DIS] 

Continuous- Variable  Model.  See:  continuous  model.  [DIS] 

Control  Station.  (1)  A  facility  which  provides  the  individual  responsible  for  controlling  the 
simulation  and  the  capability  to  implement  simulation  control  as  protocol  data  units  (PDUs) 
on  the  distributed  interactive  simulation  (DIS)  network. 

Syn:  simulation  -  management  station.  [DIS] 

D 

Data.  Representation  of  facts,  concepts,  or  instructions  in  a  formalized  manner  suitable  for 
communication,  interpretation  or  processing  by  humans  or  automatic  means.  [DIS] 

Database.  A  collection  of  data  organized  according  to  a  schema  to  serve  one  or  more 
applications.  [DIS] 

Data  Certification.  The  determination  that  data  have  been  verified  and  validated.  (1)  Data 
producer  certification  is  the  determination  by  the  data  producer  that  data  have  been  verified 
and  validated  against  documented  standards  of  criteria.  (2)  Data  user  certification  is  the 
determination  by  the  application  sponsor  or  designated  agent  that  data  have  been  verified  and 
validated  as  appropriate  for  the  specific  M&S  usage.  [DIS] 

Data  Logger.  A  device  that  accepts  protocol  data  units  (PDUs)  from  the  network  and  stores 
them  for  later  replay  in  the  same  time  sequence  as  the  PDUs  were  originally  received.  See 
also:  protocol  data  unit  (PDU).  [IEEE  1278.3] 

Data  Validation.  The  documented  assessment  of  data  by  subject  area  experts  and  comparison  to 
known  or  best-estimate  values.  (1)  Data  producer  validation  is  that  documented  assessment 
within  stated  criteria  and  assumptions.  (2)  Data  user  validation  is  that  documented 
assessment  of  data  as  appropriate  for  use  in  an  intended  M&S.  [DIS] 

Data  Verification.  The  use  of  techniques  and  procedures  to  ensure  that  data  meet  specified 
constraints  defined  by  data  standards  and  business  rules.  (1)  Data  producer  verification  is  the 
use  of  techniques  and  procedures  to  ensure  that  data  meet  constraints  defined  by  data 
standards  and  business  rules  derived  from  process  and  data  modeling.  (2)  Data  user 
verification  is  the  use  of  techniques  and  procedures  to  ensure  that  data  meet  user  speeified 
constraints  defined  by  data  standards  and  business  rules  derived  from  process  and  data 
modeling  and  that  data  are  transformed  and  formatted  properly.  [DIS] 

Data  Verification,  Validation,  and  Certification.  The  process  of  verifying  the  internal 
consistency  and  correctness  of  data,  validating  that  they  represent  real  world  entities 
appropriate  for  their  intended  purpose  or  an  expected  range  of  purposes,  and  certifying  them 
as  having  a  specified  level  of  quality  or  as  being  appropriate  for  a  specified  use,  type  of  use,  or 
range  of  uses.  The  process  has  two  perspectives:  producer  and  user  process.  See:  data 
validation,  data  verifieation,  and  data  eertification.  [DIS] 

Dead  Reckoning.  See:  remote  entity  approximation. 

Deaggregate.  See:  disaggregate.  [DIS] 
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Distributed  Interactive  Simulation  (DIS).  A  synthetic  environment  within  which  humans  may 
interact  through  simulation(s)  at  multiple  sites  networked  using  compliant  architecture, 
protocols,  standards,  and  databases  (DoDD  5000. 59P) 

E 

Electronic  Battlefleld.  See:  synthetic  environment.  [DIS] 

Entity.  Any  component  in  a  system  that  requires  explicit  representation  in  a  model.  Entities 
possess  attributes  denoting  specific  properties.  See:  simulation  entity.  [DIS] 

Environment.  (1)  The  texture  or  detail  of  the  domain,  such  as  cities,  farmland,  sea  states,  etc. 

(2)  The  external  objects,  conditions,  and  processes  that  influence  the  behavior  of  a  system 
(such  as  terrain  relief,  weather,  day,  night,  terrain  cultural  features,  etc.)  [DIS] 

Event.  (1)  An  occurrence  that  causes  a  change  of  state  in  a  simulation.  See  also:  conditional 
event;  time-dependent  event.  (2)  The  instant  in  time  at  which  a  change  in  some  variable 
occurs.  [DIS] 

Event-Driven  Simulation.  See:  event-oriented  simulation.  [DIS] 

Event-Oriented  Simulation.  A  simulation  in  which  attention  is  focused  on  the  occurrence  of 
events  and  the  times  at  which  those  events  occur;  for  example,  a  simulation  of  a  digital  circuit 
that  focuses  on  the  time  of  state  transition.  Syn:  event-driven  simulation;  event-sequenced 
simulation.  [DIS] 

Event-Sequenced  Simulation.  See:  event-oriented  simulation.  [DIS] 

Exercise.  (I)  One  or  more  sessions  with  a  common  objective  and  accreditation.  (2)  The  total 
process  of  designing,  assembling,  testing,  conducting,  evaluating,  and  reporting  on  an  activity. 
See:  simulation  exercise.  Syn:  experiment,  demonstration.  [DIS,  IEEE  1278.3] 

F 

Fidelity.  (1)  The  similarity,  both  physical  and  functional,  between  the  simulation  and  that  which 
it  simulates.  (2)  A  measure  of  the  realism  of  a  simulation.  (3)  The  degree  to  which  the 
representation  within  a  simulation  is  similar  to  a  real-world  object,  feature,  or  condition  in  a 
measurable  or  perceivable  manner.  See  also:  model/simulation  validation.  [DIS,  IEEE  1278.1] 

Field.  (1)  A  series  of  contiguous  bits,  treated  as  an  instance  of  a  particular  data  type,  that  may  be 
part  of  a  higher  level  data  structure.  (2)  An  external  operating  area  for  actual  vehicles  or  live 
entities.  See:  field  instrumentation.  [DIS,  IEEE  1278.1] 

G 

Graphical  Model.  A  symbolic  model  whose  properties  are  expressed  in  diagrams.  For  example, 
a  decision  tree  used  to  express  a  complex  procedure.  Contrast  with:  mathematical  model; 
narrative  model;  software  model;  tabular  model.  [DIS] 

Ground  Truth.  The  actual  facts  of  a  situation  without  errors  introduced  by  sensors  or  human 
perception  and  judgment.  [DIS] 

H 
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Human-in-the-Loop  Model.  See:  interactive  model. 

Human-Machine  Simulation.  A  simulation  carried  out  by  both  human  participants  and 
computers,  typically  with  the  human  participants  asked  to  make  decisions  and  a  computer 
performing  processing  based  on  those  decisions.  [DIS] 

I 

Interactive  Model.  A  model  that  requires  human  participation.  Syn:  human-in-the-loop  model. 
[DIS] 

Interoperable.  Two  or  more  simulations  are  DIS  interoperable  for  a  given  exercise  if  they  are 
DIS  compliant,  DIS  compatible,  and  their  performance  characteristics  support  a  fair  fight  to 
the  fidelity  required  for  the  exercise.  Contrast  with:  compatible,  compliant.  [DIS] 

Interoperability.  (1)  The  ability  of  a  set  of  simulation  entities  to  interact  with  an  acceptable 
degree  of  fidelity.  The  acceptability  of  a  model  is  determined  by  the  user  for  the  specific 
purpose  of  the  exercise,  test,  or  analysis.  (2)  The  ability  of  a  set  of  distributed  interactive 
simulation  applications  to  interact  through  the  exchange  of  protocol  data  units.  [DIS] 

L 

Live  Entity.  A  perceptible  object  that  can  appear  in  the  virtual  battlespace  but  is  unaware  and 
non-responsive  (either  by  intent,  lack  of  capability  or  circumstance)  to  the  actions  of  virtual 
entities.  See  also:  field  instrumentation.  Contrast  with:  live  instrumented  entity.  [DIS] 

Live  Instrumented  Entity.  A  physical  entity  that  is  in  the  real  world  and  can  be  represented  in 
the  distributed  interactive  simulation  (DIS)  virtual  battlespace  whieh  can  be  manned  or 
unmanned.  The  live  instrumented  entity  has  internal  and/or  external  field  instrumentation  (FI) 
devices/systems  to  record  and  relay  the  entity’s  surroundings,  behavior,  and/or  reaction  to 
events.  If  the  FI  provides  a  two-way  link,  the  events  that  affect  the  hve  instrumented  entity 
can  be  occurring  in  the  virtual  battlespace  as  well  as  the  real  world.  See  also:  field 
instrumentation,  live  entity.  [DIS] 

Local  Area  Network  (LAN).  A  class  of  data  network  which  provides  high  data  rate 
interconnection  between  network  nodes  in  close  physical  proximity.  [IEEE  1278.3] 

M 

Measure  of  Performance  (MOP).  Measure  of  how  the  system/individual  performs  its  functions 
in  a  given  environment  (e.g.,  number  of  targets  detected,  reaction  time,  number  of  targets 
nominated,  susceptibility  of  deception,  task  completion  time).  It  is  closely  related  to  inherent 
parameters  (physical  and  structural)  but  measures  attributes  of  system  behavior.  See  also: 
measures  of  effectiveness  (MOE).  [lEE  1278.3] 
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Model.  (1)  An  approximation,  representation,  or  idealization  of  selected  aspects  of  the  structure, 
behavior,  operation,  or  other  characteristics  of  a  real-world  process,  concept,  or  system. 

Note:  Models  may  have  other  models  as  components.  (2)  To  serve  as  a  model  as  in  (1).  (3) 

To  develop  or  use  a  model  as  in  (1).  (4)  A  mathematical  or  otherwise  logical  representation  of 
a  system  or  a  system’s  behavior  over  time.  [DIS] 

Model/Simulation  Accreditation.  The  official  certification  that  a  model  or  simulation  is 
acceptable  for  use  for  a  specific  purpose.  See  also:  distributed  simulation  accreditation. 
Contrast  with:  model/simulation  validation,  model/simulation  verification.  [DoDD  5000.59] 

Model/Simulation  Validation.  The  process  of  determining  the  degree  to  which  a  model  is  an 
accurate  representation  of  the  real  world  from  the  perspective  of  the  intended  use(s)  of  the 
model.  See  also:  distributed  simulation  validation,  fidelity.  Contrast  with:  model  simulation 
accreditation,  model  simulation  verification.  [DoDD  5000.59] 

Model/Simulation  Verification.  The  process  of  determining  that  a  model  implementation 
accurately  represents  the  developer's  conceptual  description  and  specifications.  See  also: 
distributed  simulation  verification.  Contrast  with:  model  simulation  accreditation,  model 
simulation  validation.  [DoDD  5000.59] 

N 

Network  Filter.  A  system  to  selectively  accept  or  reject  data  received  from  the  network.  [DIS] 

Network  Node.  A  specific  network  address.  See:  node.  Contrast  with:  processing  node.  [DIS] 

Node.  A  general  term  denoting  either  a  switching  element  in  a  network  or  a  host  computer 
attached  to  a  network.  See:  processing  node;  network  node.  [IEEE  1278.1,  IEEE  1278.2] 

o 

Operational  Environment.  A  composite  of  the  conditions,  circumstances,  and  influences  which 
affect  the  employment  of  military  (or  other)  forces  and  the  decisions  of  the  unit  commander  or 
person  in  charge.  [DIS] 

P 

Platform.  A  generic  term  used  to  describe  a  level  of  representation  equating  to  vehicles,  aircraft, 
missiles,  ships,  fixed  sites,  etc.,  in  the  hierarchy  of  representation  possibilities.  Other 
representation  levels  include  units  (made  up  of  platforms)  and  components  or  modules  (which 
make  up  platforms.)  [DIS] 

Protocol  Data  Unit  (PDU).  A  DIS  data  message  that  is  passed  on  a  network  between  simulation 
applications  according  to  a  defined  protocol.  [IEEE  1278.1] 
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R 

Real  Time.  In  modeling  and  simulation,  simulated  time  advances  at  the  same  rate  as  actual  time; 
for  example,  running  the  simulation  for  one  second  results  in  the  model  advancing  time  by  one 
second.  Contrast  with:  fast  time,  slow  time.  [DIS] 

Resolution.  (1)  The  degree  to  which  near  equal  results  values  can  be  discriminated.  (2)  The 
measure  of  the  ability  to  delineate  picture  detail.  [DIS] 

s 

Scenario.  (1)  Description  of  an  exercise  (initial  conditions).  It  is  part  of  the  session  database 
which  configures  the  units  and  platforms  and  places  them  in  specific  locations  with  specific 
missions.  (2)  An  initial  set  of  conditions  and  time  line  of  significant  events  imposed  on  trainees 
or  systems  to  achieve  exercise  objectives.  See:  field  exercise.  [DIS,  IEEE  1278.3] 

SIMNET  (Simulator  Networking).  The  prototype  distributed  simulation  upon  which  DIS  was 
based.  [DIS] 

Simulate.  To  represent  a  system  by  a  model  that  behaves  or  operates  like  the  system.  See  also: 
emulate.  [DIS] 

Simulated  Time.  Time  as  represented  within  a  simulation.  Syn:  virtual  time.  See  also:  fast  time; 
real  time;  slow  time.  [DIS] 

Simulation.  (1)  A  model  that  behaves  or  operates  like  a  given  system  when  provided  a  set  of 
controlled  inputs.  Syn:  simulation  model.  See  also:  emulation.  (2)  The  process  of  developing 
or  using  a  model  as  in  (1).  (3)  An  implementation  of  a  special  kind  of  model  that  represents  at 
least  some  key  internal  elements  of  a  system  and  describes  how  those  elements  interact  over 
time.  [DIS] 

Simulation  Environment.  (1)  Consists  of  the  natural  physical  environment  surrounding  the 
simulation  entities  including  land,  oceans,  atmosphere,  near-space,  and  cultural  information. 
(2)  All  the  conditions,  circumstances,  and  influences  surrounding  and  affecting  simulation 
entities  including  those  stated  in  (1).  [DIS] 

Simulation  Exercise.  An  exercise  that  consists  of  one  or  more  interacting  simulation 
applications.  Simulations  participating  in  the  same  simulation  exercise  share  a  common 
identifying  number  called  the  exercise  identifier.  These  simulations  also  utilize  correlated 
representations  of  the  synthetic  environment  in  which  they  operate.  See:  live  simulation. 

[IEEE  1278.1,  IEEE  1278.2] 

Simulation  Fidelity.  Refers  to  the  degree  of  similarity  between  the  simulated  situation  and  the 
operational  situation.  [IEEE  1278.3] 

Simulation  Time.  (1)  A  simulation’s  internal  representation  of  time.  Simulation  time  may 
accumulate  faster,  slower,  or  at  the  same  pace  as  real  time.  (2)  The  reference  time  (e.g., 
universal  coordinated  time)  within  a  simulation  exercise.  This  time  is  established  ahead  of 
time  by  the  simulation  management  function  and  is  common  to  all  participants  in  a  particular 
exercise.  [DIS,  IEEE  1278.1] 
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Simulator.  (1)  A  device,  computer  program,  or  system  that  performs  simulation.  (2)  For 
training,  a  device  which  duplicates  the  essential  features  of  a  task  situation  and  provides  for 
direct  practice.  (3)  For  distributed  interactive  simulation  (DIS),  a  physical  model  or  simulation 
of  a  weapons  system,  set  of  weapon  systems,  or  piece  of  equipment  which  represents  some 
major  aspects  of  the  equipment’s  operation.  [DIS] 

Site.  (1)  An  actual  physical  location  at  a  specific  geographic  area,  e.g.,  the  Fort  Knox  Close 
Combat  Test  Bed  (CCTB).  (2)  A  node  on  the  network  used  for  distributed  simulation  such  as 
the  Defense  Simulation  Internet  (DSI)  long  haul  network.  (3)  A  level  of  configuration 
authority  within  a  DIS  exercise.  [DIS] 

V 

Validation.  See:  data  validation,  distributed  simulation  validation,  face  validation, 
model/simulation  validation.  [DIS] 

Verification.  See:  data  verification,  distributed  simulation  verification,  model/simulation 
verification 

Verification  and  Validation  (V&V)  Proponent.  The  agency  responsible  for  ensuring  V&V  is 
performed  on  a  specific  model  or  simulation.  [DIS] 

Vignette.  A  self-contained  portion  of  a  scenario.  [DIS] 

Virtual  Battlespace.  The  illusion  resulting  from  simulating  the  actual  battlespace.  [DIS] 

w 

War  Game.  A  simulation  game  in  which  participants  seek  to  achieve  a  specified  military 

objective  given  pre-established  resources  and  constraints;  for  example,  a  simulation  in  which 
participants  make  battlefield  decisions  and  a  computer  determines  the  results  of  those 
decisions.  See  also:  management  game.  Syn:  constructive  simulation;  higher  order  model 
(HOM).  [DIS] 

Wide  Area  Network  (WAN).  A  communications  network  of  devices  which  are  separated  by 
substantial  geographical  distance.  Syn:  long  haul  network.  [IEEE  1278.3] 
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APPENDIX  D  “  List  of  Acronyms 


4  ID 

4th  Infantry  Division,  Fort  Hood,  Texas 

ADA 

air  defense  artillery 

ADS 

advanced  distributed  simulation 

AFATDS 

Advanced  Field  Artillery  Tactical  Data  System 

AFB 

Air  Force  base 

ALQ-131 

a  mature  self-protection  jammer  system;  an  electronic  countermeasures 
system  with  reprogrammable  processor  developed  by  Georgia  Technical 
Research  Institute 

AM 

amplitude  modulation 

ANIU 

air  network  interface  unit 

ARIES 

Advanced  Radar  Imaging  Emulation  System 

ARPA 

Advanced  Research  Projects  Agency 

ASAS 

All  Source  Analysis  System 

ATACMS 

Army  Tactical  Missile  System 

A-tracker 

automatic  tracker 

ATWS 

Advanced  Technology  Work  Station 

Bde 

brigade 

Bn 

battalion 

C4I 

command,  control,  communications,  computers  and  intelligence 

C4ISR 

command,  control,  communications,  computers,  intelligence,  surveillance 
and  reconnaissance 

CAMPS 

Compartmented  AH  Source  Analysis  System  (ASAS)  Message  Processing 
System 

CBS 

corps  battlefield  simulation 

CDP 

central  data  processor 

CEP 

circular  error  probability 

CGS 

common  ground  station 

Co 

company 

COI 

critical  operational  issue 

COT&E 

contingency  operations  test  and  evaluation 

CPU 

central  processing  unit 

D&SA  BL 

Depth  and  Simultaneous  Attack  Battle  Lab 

DCT 

digital  communications  terminal 

DDSA 

deputy  director,  system  assessment 

DIS 

distributed  interactive  simulation 

DMAP 

data  management  and  analysis  plan 

DoD 

Department  of  Defense 

DT&E 

developmental  test  and  evaluation 

ECCM 

electronic  counter-countermeasure 

ESPDU 

entity  state  protocol  data  unit 

ETE 

End-to-End  Test 

EW 

electronic  warfare 

FDC 

fire  direction  center 
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FI 

FM 

FTI 

FTP 

GHQ 

GPS 

GSM 

GSMR 

HF 

HLA 

hrs 

ICD 

ID 

IEEE 

IPX 

JADS 

Janus 

Joint  STARS 

JPO 

JT&E 

JTF 

km 

LAN 

LFP 

LGSM 

LSP 

M&IS 

M&S 

MB 

Mbps 

MI 

mm 

MOE 

MOP 

MOT&E 

ms 

MTI 

N&E 

NC 

NETVisualizer^'^ 

NIU 

NTP 

O&C 

OSD 


functionality  and  integration 
frequency  modulation 
fixed  target  indicator 
file  transfer  protocol 
general  headquarters 
global  positioning  system 
ground  station  module 

Ground  Station  Module  Replicator,  Fort  Huachuca,  Arizona 

high  frequency 

high  level  architecture 

hours 

interface  control  document 

infantry  division;  identification 

Institute  of  Electrical  and  Electronics  Engineers 

integrated  product  team 

Joint  Advanced  Distributed  Simulation,  Albuquerque,  New  Mexico 

interactive,  computer-based  simulation  of  combat  operations 

Joint  Surveillance  Target  Attack  Radar  System 

joint  program  office 

joint  test  and  evaluation 

joint  test  force 

kilometers 

local  area  network 

Live  Fly  Phase 

light  ground  station  module 

Linked  Simulators  Phase 

management  and  integration  software 

modeling  and  simulation 

megabyte 

megabits  per  second 
military  intelligence 
millimeter 

measure  of  effectiveness 

measure  of  performance 

multiservice  operational  test  and  evaluation 

millisecond 

moving  target  indicator 
network  and  engineering 
network  coordinator 

software  that  displays  real-time  bandwidth  use  in  a  rolling  bar  graph  format 

for  quick  visual  reference 

network  interface  unit 

network  time  protocol 

operations  and  control 

Office  of  the  Secretary  of  Defense 
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OT 

operational  test 

OT&E 

operational  test  and  evaluation 

OWS 

operator  workstation 

PDU 

protocol  data  unit 

PM 

program  manager 

POC 

point  of  contact 

PTP 

program  test  plan 

PVD 

plan  view  display 

RPSI 

radar  processor  simulator  and  integrator 

RSR 

radar  service  request 

RWS 

remote  workstation 

SAR 

synthetic  aperture  radar 

SCDL 

surveillance  control  data  link 

SE 

synthetic  environment 

sec 

second 

SGI 

Silicon  Graphics,  Inc. 

SIT 

System  Integration  Test 

SM&C 

system  management  and  control 

SME 

subject  matter  experts 

Spectrum 

an  instrumentation  suite  used  to  measure  bandwidth  utilization 

STARS 

surveillance  target  attack  radar  system 

STRICOM 

U.S.  Army  Simulation,  Training,  and  Instrumentation  Command 

SUT 

system  under  test 

SWA 

Southwest  Asia 

T&E 

test  and  evaluation 

T-1 

digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1.544  megabits 
per  second 

TAG 

target  analysis  cell 

TAFSM 

Tactical  Army  Fire  Support  Model 

TCAC 

Test  Control  and  Analysis  Center,  Albuquerque,  New  Mexico 

TEMPEST 

special  shielding  against  electromagnetic  radiation 

TEXCOM 

U.S.  Army  Test  and  Experimentation  Command 

TRAC 

U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UAV 

unmanned  aerial  vehicle 

UDP 

user  data  protocol 

UHF 

ultra  high  frequency 

V&V 

verification  and  validation 

VHP 

very  high  frequency 

VSTARS 

Virtual  Surveillance  Target  Attack  Radar  System 

YV&A 

verification,  validation,  and  accreditation 

WAN 

wide  area  network 

WSMR 

White  Sands  Missile  Range 

XPATCHES 

E-8C  synthetic  aperture  radar  simulation  developed  by  Wright  Laboratory, 
Dayton,  Ohio,  and  Loral  Defense  Systems,  Goodyear,  Arizona 
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